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abstract “ — — 

i he Texarkana model of performance contracting consists of rive major ei 
(1) turnkeying of cost-effective new technology as a basic program goal 
use of a performance contract for instruction, (3) use of a management support 

° r> r • select,on of th ® learning system contractor by formal competition, 
and L; use of independent evaluators and auditors. It Is clear from the 
Texarkana experience in 1970-71 that performance contracting has been a mechanism 
or educational process- change. In both years, the program successfully met the 
goal of reducing the dropout rate. It was unsuccessful, however, in producing any 
significant achievement gains as measured by standardized norm-referenced tests. 
Achievement measurement still has many logistic, administrative and conceptual 
problems. Texarkana evaluation procedures have enabled Impressive feedback of 
pertinent managerial information. Model Cities provided a mechanism for solving 
some problems and a source, of seed money for modest expansion and development of 
the program. For related documents, see ED 056247, 248, 250,2 51, and 252. (Author/ 
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PREFACE 



This Report is a product of Rand’s study of performance contracting in educa- 
tion. The study is sponsored by the Assistant Secretary for Plaxining and Evaluation, 
U.S. Department of Health, Education and Welfare, under Contract No. HEW-03- 
70-156. 

Case Studies in Educational Performance Contracting comprises six volumes. 
Each is a self-contained study; together they provide a multifaceted view of perform- 
ance contracting. The six volumes are: 



1. R'900/l-HEW, Conclusions and Implications, by P. Carpenter and 
G. R. Hall 

2. R-900/2-HEW, Norfolk, Virginia, by P. Carpenter 

8. R'900/3-HEW, Texarkana, Arkansas and Liberty-Eylau, Texas, by 

P. Carpenter, A. W. Chalfant, and G. R, Hall 

4, R-900/4-HEW, Gary, Indiana, by G. R. Hall and M. L, Rapp 

5, R-900/5-HEW, Gilroy, California, by M, L. Rapp and G, R. Hall 

6, R-90U/6-HEW, Grand Rapids, Michigan, by G. C, Sumner 

This study is the second of three Rand Reports on the subject. The first Report 
was J, P, Stucker and G, R. Hall, The Performance Contracting Concept in Education, 
The Rand Corporation, R-699/1-HEW, May 1971, The third Report will be a per- 
formance contracting guide intended for use by educational officials. 



SUMMARY 



The Texarkana performance contracting program is historically important and 
has been a model for other programs. The Texarkana model consists of five major 
elements: (1) turnkeying of cost-effective new technology as a basic program goal; (2) 
use of a performance contract for instruction; (3) use of a management support 
contractor; (4) selection of the learning system contractor by formal competition; 
and (5) use of independent evaluators and auditors. 

The Texarkana program also is instructive because it is the first district to 
attempt to "turnkey” (adopt for school system use) a performance contractor’s sys- 
tem. This study gives particular attention to this aspect of performance contracting 
in Texarkana. 

Looking back on the 1969-70 program, we can derive four general implications 
for performance contracting. First, it is difficult to measure achievement. Beyond 
the need to counteract any incentives for "teaching to the test,’ - testing require- 
ments impose serious administrative burdens. There are also serious problems of 
test validity and test reliability. 

Second, programs should make provisions for settling contract disputes when 
the original conditions envisioned do not apply. In Texarkana in 1969-70, the prob- 
lem was the lack of valid test data; other districts have run up against other prob- 
lems. Some fall-back system for .contract settlement seems highly desirable. 

Third, performance contracting creates new responsibilities and work require- 
ments for school districts. These programs involve the school district as well as the 



contractor. If other organizations are also engaged in management support or 
evaluation, coordination and role-definition can become important. 

Fourth, change is endemic to performance contracting. One of the strongest 
features of the Texarkana program is that it was planned for five years, so that 
development was possible. Using a new contractor and adding new programs beyond 
turnkey were major changes in 1970-71. In 1971-72 the program was changed fur- 
ther; instead of using private contractors to manage the learning centers, the school 
districts involved will take over their operation. 

It is clear from the Texarkana experience in 1970-71 that performance contract- 
ing has been a mechanism for educational process change. Also, as an adjunct to the 
program, Texarkana has begun to develop program budgets and a coordinated man- 
power education program. The Rapid Learning Centers and the turnkey classrooms 
differ from the conventional Texarkana classrooms in materials and procedures. 
The turnkey classrooms, however, embody the 1969-70 Dorset! system in only the 
most general fashion. We expect performance contracting will lead to a much better 
compensatory education program in Texarkana and perhaps even to a general 
curriculum improvement. When the turnkey process is completed, however, we 
expect that classrooms will only indirectly reflect the technology and procedures 
used by the performance contractors. 

In both years, the Texarkana program successfully met the goal of reducing the 
dropout rate. It was unsuccessful, however, in producing any significant achieve- 
ment gains as measured by standardized norm-referenced tests. As a result, the 
school districts involved accepted the evaluator’s recommendation and assumed 
direct operation of the learning centers. LEA control of these centers for 1971-72 will 
give Texarkana the chance to see if it has gained enough experience locally to do 
better than the private contractors. 

Achievement measurement still has many logistic, administrative, and concep- 
tual problems. Texarkana attempted to attack the conceptual problems in 1970-71 
by using criter ion-referenced tests for part of the payment to the contractor. Their 
experience indicates that the state of this art calls for considerable development. 

Texarkana evaluation procedures have been a strong aspect of the program. In 
particular, feedback of pertinent managerial information was impressive. In 1970- 
71, the link between evaluation and management was strengthened by giving the 
evaluating organization a management support contract as well. 

Lines of authority and responsibility continued to be complex. The result was 








vi 



that some parts of this multiobjective program received less attention than they 
needed. 

Model Cities continued to figure importantly in the program. It has provided not 
only a mechanism for solving some problems, but also a source of seed money for 
modest expansion and development of the program. It has further helped to inte- 
grate the dropout prevention program with the other city efforts in education and 
manpower training. 
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I. INTRODUCTION 



TEXARKANA AND THE BIRTH OF PERFORMANCE CONTRACTING 

In any discussion of performance contracting, the Texarkana project must as- 
sume a prominent place. Historically, it pioneered the performance contracting 
technique; 1 it aroused nationwide interest in and controversy about performance 
contracting; and it was the locale of the first scandal over "teaching to the test” and 
the first dispute between a local education agency (LEA) and learning system con- 
tractor (LSC) over final payment. 

The project is important also because it has served as a model for other perform- 
ance contracting programs. True, Texarkana is not the only such model, but it 
incorporated many features advocated by authorities at the U.S. Office of Education 
and elsewhere who are concerned with educational accountability and technological 
change. In particular, the procedures reflected the theoretical concepts of Leon M. 
Lessinger, formerly of the United States Office of Education (USOE), and Charles L. 
Blaschke and his associates at Education Turnkey Systems, Inc. Texarkana proce- 
dures have been reflected in the Office of Economic Opportunity’s performance 
contracting experiment involving 20 performance contracts, and in programs in 
Virginia, Dallas, Texas, Jacksonville, Florida, and other cities. 

1 During 1969-70, there were also some small performance contracting programs in Portland, Oregon, 
but these did not receive the public attention given Texarkana. 



Texarkana’s experience is particularly instructive because it has been involved 
with performance contracting for two school years, and has attempted to "turnkey” 
systems— that is, convert contractor-developed systems into regular school-district- 
managed programs. Since most school districts seek to turnkey successful perform- 
ance contracting programs, this facet of the Texarkana program will receive special 
attention. 

The plan of this Report is as follows: The remainder of this introduction pre- 
sents some notable background data on Texarkana. Section II describes the five-year 
dropout-prevention program and the start of performance contracting in 1969-70, 
Section III describes performance contracting in 1970-71, discussing both the turn- 
key classrooms and the contractor-managed centers. The impacts of performance 
contracting on educational processes, products, and program participants will be 
considered in each section. Section IV deals with future plans, and Section V draws 
overall conclusions from the Texarkana experience. 




BACKGROUND DATA ON TEXARKANA 

According to one popular version of Texarkana’s origin Colonel Gus Knobel, a 
railroad surveyor, nailed a board with the sign "TEX-ARK-ANA” onto a tree at the 
border of Texas and Arkansas and proclaimed, "This is the name of the town which 
is to be built here,” He derived the name from the three adjacent states of Texas, 
Arkansas, and Louisiana, believing Louisiana to be only a few miles away (it was 
actually 33 miles), 8 As can be seen in Fig, 1, Texarkana is a crossroads town. It has 
three railways and four major highways going to such nearby cities as Shreveport 
and Little Rock, with Dallas and Oklahoma City a little bit farther. The Texas- 
Arkansas border, running down the main street through the post office, is a tourist 
attraction, 

Texarkana had an estimated 1969 population of about 60,000, with 24,000 in 
Arkansas and 36,000 in Texas, Its area is roughly 25,1 square miles,® Although 
cotton used to be a sizable crop, its importance has declined along with agriculture 
in general. The economy is dominated by ammunition and ordnance manufacture 

* Today in Texarkana, Texarkana Chamber of Commerce, 1970, p. 19. 

3 Texarkana Chamber of Commerce, 
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at the Lone Star Army Ammunition Plant and the Red River Army Depot. Only 
seven other manufacturers employ over 100 people (tires, mobile homes, pickles, 
paper mill, etc.). 3 Retail and wholesale trade is the other main source of employ- 
ment; Texarkana has 150 wholesale businesses and serves a 19-county retail trade 
area. 4 

Texarkana’s main concerns are employment and integration. Its current unem- 
ployment rate is above 6 percent and there is little economic diversification. The 
immediate problem is how to adjust to the wind-down of the Vietnam war. 3 Loth the 
Chamber of Commerce and the Model Cities Demonstration Agency have projects 
under way to attract new industry, but results will take some time. Th ~ region is 
not prosperous; more than 30 percent of the school pupils come from families earn- 
ing less than $2000 per year. 8 

Texarkana is served by three school districts, each having about one-third black 
students. Texarkana, Texas, Independent School District No. 19-907, and Texar- 
kana, Arkansas, School District No. 7 are about the same size — some 6900 students 
each. Liberty-Eylau, Texas, RHSD No. 708, a rural district partly contained in 
Texarkana, has about 2500 students. Each district has an elected school board and 
an appointed superintendent. Some basic data for the three districts are presented 
in Table 1. Only Texarkana, Arkansas, and Liberty-Eylau (pronounced "eye-loo”) 
participated in the performance contracting program. 

The budget for the urban Texarkana schools in 1970-71 was $2.7 million for 
current expenditures and $5,5 million for all expenditures including construction. 
The budget for Liberty-Eylau was $1.2 million for current expenditures and $1.3 
million total. For Texarkana, Arkansas, this amounts to a per-pupil expenditure of 
about $440 compared with the national average of more than $765 per pupil. 6 

Texarkana, Arkansas, has nine schools: six elementary, two junior highs, and 
one senior high. One of the elementary schools is the former all-black junior high; 
following integration, it has been used for the sixth grade alone. The all-black 
elementary school was converted to special education and kindergarten classes. 
Liberty-Eylau has six schools serving the following grade levels: one K-2, one third 
grade and special education, one 4-6, one 7-8, one 9-10, and one 11-12. This unusual 
arrangement resulted partly from integration, with the black senior high becoming 

4 Today in TexarkanQi p. 10, 

5 R. A. Bumstead, "Texarkana: The First Accounting,” Educate f VoL 8, No. 2, March 1070, p. 27. 

6 Ibid. 




ENROLL! IENT AND STAFF FOR TEXARKANA DISTRICTS 
IN 1969-70 Aim 1970-71 



Item 


Texarkana 5 
Arkansas 


Liberty— 
Eylau ? 
Texas 


Texarkana, 

Texas 


Enrollment 








1969-70 


6929 a 


2489 


6962 


1970-71 


6959 


2580 


6895 


Percent black 


33.6% 


N.A.b 


30% 


Professional staff 








1969-70 


300 


N,A, 


356 


1970-71 


310 


150 


363 


Percent black 


21 % 


N.A, 


30% 



Computed from the average dally attendance of 
6062, using the 1970-71 ratio of 87% between ADA 
and enrollment. 



N.A. = not available. 



the new integrated junior high, and the white junior high becoming the 9-10 grade 
school. 7 

The second major issue in Texarkana is school integration. As of the 1960 
census, Texarkana was 73 percent white and 27 percent black, with most of the 
blacks segregated in just a few census tracts. Reflecting this situation, there were 
several all-black schools at all levels that served the majority of black students. By 
early 1969, total integration had just begun at Texarkana’s Texas High School, and 
was slated for all the schools over the next few semesters. Widespread integration 
was scheduled for Arkansas for the 1969-70 school year, 

Texarkana, Arkansas, uses three tracks (advanced, regular, and basic) in most schools, Liberty- 
iiylau tried tracking in 1970-71, but several pringipals said they would switch back in 1971-72. The only 
remedial reading classes are at the elementary schools. Both districts have vocational programs with 
Arkansas dapending on Model Cities funds. 



Integration on the Texas side of the border had been accompanied by a serious 
increase in school dropouts. This was attributed to the wide disparity betweer 
educational achievement of whites and blacks in Texarkana. Consequently, the 
Texarkana, Arkansas, school officials felt it imperative that a special effort be made 
to prevent integration from leading to severe academic difficulties and widespread 
dropouts. At the same time, Texarkana’s Model Cities Agency was exploring possible 
programs and funding sources. The result w as the development of a five-year drop- 
out-prevention program and the first large performance contract. 

At the beginning, it was intended that all three districts would be involved in 
the dropout-prevention program that formed the context for performance contract- 
ing for educational services. Texarkana, Texas, dropped out of the consortium be- 
cause of problems associated with racial desegregation guidelines, leaving Texar- 
kana, Arkansas, and Liberty-Eylau, Texas, as partners. Texarkana, Arkansas is 
designated as the manager and the fiscal agent for the partnership. Unless otherwise 
stated, a reference to Texarkana includes Liberty-Eylau, Texas. 




II. THE TITLE VIII DROPOUT PREVENTION PROGRAM, 

1969-70 



INTRODUCTION 

The genesis of performance contracting in Texarkana, the Borsett-Texarkana 
contract, and the operation of the Dorsett Rapid Learning Centers (RLCs) have been 
discussed often,® Here we summarize the program and discuss some aspects that 
merit special attention. 



• Among the leading accounts are: „ , 1Q7n „ 94 q 7 

Etichard A. Bumstead, "Texarkana, The First Accounting,” Educate, Vol. 3, No. 2, March 1970, pp. 24-37. 
L Dorsett, "Interview,” [Reading Newsreport,] Vol. 4, No. 2, November/pecember 1969. 

Education Turnkey Systems, Inc., [Performance Contracting in Education,] Research Press, Champ _gn, 

3tanley°Eiam, "The Age of Accountability Dawns in Texarkana," [Phi Delta Kappan,] June 1970, pp. 

M. j! Filogamo, "New Angle on Accountability,” [Today’s Education,] Vol. 69, . No. 5, May l970 p. 53. 
Leon M. Lessinger, "Accountability in Public Education,” [Today s Education,] Vol. 59, No. 5, May 1970, 

—^ P ‘" After Texarkana, What?”, [Nation’s Schools,] Vol. 84, No. 6, December 1969, 37-40. 

— -, [Every Kid a Winner: Accountability in Education,] Simon and Schuster, New York, 1970, 
"Performance Contracting as Catalyst for Reform,” [Educational Technology,] August 1969, pp. 5-9. 
"Texarkana First,” [Education Turnkey News,] Vol. 1, No. 1, April 1970, pp. ^T^CAlmost e *® r y ® & 

quent issue of [Education Turnkey News] has contained reports on, or related to, the Texarkana 

Edward* Willingham, "Education Report/ Performance Contesting in School 

’Accountability’ Idea,” [National Journal,] Vol. 2, No. 43, October 24, 1970, pp. 2324-23 . 




START OF THE PROGRAM 

In the fall of 1968, all three Texarkana districts were involved in planning for 
imminent full integration after some previous efforts in the junior highs at the 
ninth-grade level. The city had also acquired an active Model Cities Agency funded 
by the U.S. Department of Housing and Urban Development, which included a 
manpower and education component as one of its six major programs, and the 
agency was interested in means for educational reform. Dr, Joel Hart of the Univer- 
sity of Arkansas, who was also associated with the Institute for Politics and Plan- 
ning in Washington, D,0,, was assisting Model Cities in teaching leadership skills 
in the Model Cities neighborhoods. 9 Hart had formerly worked with Charles L. 
Blaschke at the Institute, trying to set up a performance contract for young adults 
in Georgia. He also knew that a recent amendment to the Elementary and Second- 
ary Education Act of 1965 (ESEA) had authorized money for comprehensive demon- 
stration dropout-prevention programs. With a key problem in Texarkana integra- 
tion being the wide disparity in achievement levels between black and white stu- 
dents, he thought a performance contract aimed at teaching basic skills to the lowest 
achievers as part of a full dropout-prevention program might be a solution. 

Hart contacted Blaschke in early December 1968. Blaschke came to Texarkana 
at once, knowing that the deadline for planning grants for the dropout prevention 
program was just a week away. Blaschke went from the Model Cities office to the 
Arkansas district’s coordinator of Federal programs, and thence to the superintend- 
ents on both the Arkansas and Texas sides of town and in Liberty-Eylau, The 
superintendents were not too happy about bringing in so many outsiders, but the 
need was pressing, so the school boards were briefed and they authorized develop- 
ment of the project. Blaschke’s original work was covered under a contract with the 
Institute, reproduced in Appendix A. Strong pressure for integration was coming 
from HEW’s threat to cut off Federal funds, especially the large amounts from 
"impacted area” relief (due to the large Army installations in the area) and Title 
I of ESEA (compensatory education). Given Texarkana’s low level of school expendi- 
tures and its economic problems, this threat was serious. 

In a Preliminary Proposal to USOE in mid-December, Texarkana stated that 
integration could be expected to produce a serious dropout problem. The dropout 
figures during the 1867-68 school year were 6 percent for Texarkana, Arkansas’ 
program schools, and 4 percent for Liberty-Eylau’s program schools. The proposal 

B Bumstead, op, cit,, p. 26. 







pointed out the conservative nature of the estimate due to an estimated large 
number of children of low-skilled families who never enroll. Most of the Model 
Neighborhood 10 schools are heavily black, and for the Arkansas district there was 
a median difference of 70 percentile points in scores on standardized achievement 
tests between the schools inside the Model Neighborhood and those outside. The 
proposal cited studies showing that academic deficiencies were the principal cause 
of dropouts, and argued f h- >t integration would force the Model Neighborhood stu- 
dents to try to compete with a much more accomplished peer group. Data cited by 
Bumstead sharply illustrate the need for remedial training: 

For example, consider the achievement levels of students at the three junior 
high schools in Texarkana, Arkansas. These schools are yet to be completely 
integrated [1968-69], In reading achievement, students at the Jefferson Ave- 
nue Junior High School, a predominantly white, middle-class school, ranked 
in the 75th percentile on the Iowa Tests of Basic Skills. At College Hill 
Junior High School, where the races are about equally represented and 
where family incomes are somewhat lower, student achievement falls to the 
20th percentile. At Washington Junior High School, an all-black school 
located in a Model Cities neighborhood, students read at the second percen- 
tile — yes, the second percentile. 

In all areas of achievement, the average grade level for ninth graders at 
Washington was 6.4, at Jefferson Avenue 9.4, as measured by the Iowa Tests 
of Basic Skills. 11 

The Office of Education awarded a planning grant to enable Texarkana to de- 
velop a formal proposal for a full five-year dropout-prevention plan. There are three 
noteworthy features of this preliminary proposal. First, it was conceived as one 
component of a community-wide plan to bring education and manpower develop- 
ment together. This plan in turn was but one component of the major attempt at 
city renewal catalyzed by the Model Cities. Though Texarkana was the first perform- 
ance contract, and the actual program is much different from the proposal, it is still 
unusually well integrated with community efforts. While the dropout program 
focused on the academic deficiency aspects of career (not just school) problems, it was 
intended to treat causes as well as symptoms. Specifically, there was to be a quick 

10 The areas of concentrated low-income Yamilies, largely black, serving as the target population of 
the Model Cities renewal. 

11 Bumstead, op. eit, p. 27, 
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catch-up program at the junior and senior high level to stave off integration prob- 
lems, followed by remediation and then by curriculum reform at the elementary 
level, so that the students would be equipped for success from the beginning. All of 
this was planned to be absorbed into the regular schooling to produce lasting change. 

Supporting this academic side there were to have been major programs expand- 
ing vocational orientation and training courses, including centers for those who had 
already dropped out. Other efforts included an adult General Equivalency Degree 
program at the advanced levels of the school program, and Head Start and Follow- 
Through at the beginning. Given this goal-oriented and comprehensive framework 
to operate in, the dropout program has had a strong effect of its own in fostering 
coordinated efforts from the Federal, regional, school, and community levels. 

We emphasize that the original intention of the program was to be a part of a 
broad educational-manpower development effort aimed at community renewal, even 
though most public discussion of the program has focused on the narrower objectives 
of developing reading and math skills. It is important to note, however, that the 
grand design was not implemented all at once. The initial cost estimate was $750,000 
for the first year. With Texarkana, Texas eliminated, USOE was unwilling to invest 
more than $250,000. The result was elimination of the Work Study component, 
elimination of a program to develop cost-effectiveness measures, elimination of a 
program to identify potential dropouts, and a reduction of the students to be enrolled 
in the EAC from 400 to 200. 12 The program finally funded by USOE focused almost 
exclusively on what came to be called the Rapid Learning Centers (RLC). 

The second noteworthy feature of this proposal is that it reflected substantial 
inputs from Blaschke, This feature became institutionalized as the management- 
support component of the program. The Addendum to the proposal to USOE goes 
into considerable detail about the role and function of the Management Support 
Contractor, who at the start of the Texarkana program was Education Turnkey 
Systems, Inc., founded by Blaschke after he left the Institute for Politics and Plan- 
ning. 

The third feature is that the initial proposal did not discuss evaluation and 
auditing. This is noteworthy because a very important feature of the Texarkana 
program was the use of an independent evaluator and an independent auditor. 
These participants have been emphasized by many, including Lessinger (at the time 
of Texarkana one of the leading supporters of performance contracting at USOE), 



12 Ibid., pp. 28-20, The 1969-70 program eventually accommodated 381 students. 



as key elements in the use of performance contracting to achieve educational ac- 
countability. 



THE TEXARKANA MODEL 

The theory behind the program evolved from early theoretical papers of Less- 
inger, Blaschke, and others, to the initial proposal to USOE, to the final USOE 
proposal, then to the Request for Proposal sent to various learning system contrac- 
tors, and finally to the program actually instituted. The result was what we call the 
Texarkana Model, 

The Texarkana Model, as we view it, comprises five major elements: (1) turnkey- 
ing of cost-effective new technology as the basic program goal; (2) use of a perform- 
ance contract for instruction, i.e., a contract between an LEA and an LSC for 
instruction in basic skills with payment based on the results of pro- and post-tests; 
(3) use of a management support contractor to assist the LEA; (4) selection of the 
LSC by a formal competition utilizing a Request for Proposal (RFP); (5) use of 
independent contractors for evaluation and/or educational auditing. We shall 
briefly explain each major element. 



Turnkeying Cost-Effective New Technology 



The LSC s role in the Texarkana program (and most other performance con- 

icwed as transitory. As conceived, his role is to come into a 



trading programs) is vie 



school district to test out, demonstrate, and validate the cost-effectiveness of a new 
learning system. The technology is then to be transferred i ) the school district for 
in-house operation by the LEA, The performance contract is viewed as a catalyst for 
educational reform. Reform, in turn, is needed in order to achieve more fundamental 
goals such as dropout prevention, or elimination of functional illiteracy. 

An educational system or learning system consists of a set of resources and 



processes designed to yield certain defined educational results. In the Texarkana 
context, learning systems came to be discussed in terms of three dimensions: (1) new 
materials and equipment employing new methods of learning; (2) environmental 
modification; and (3) new incentives. We shall discuss these elements later in the 



section on the learning system utilized by Dorsett Educational Systems, Inc. 
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One notion behind the Texarkana program was that the problems Texarkana 
faced required highly innovative solutions. The LEA selected Dorsett Educational 
Systems, Inc., of Norman, Oklahoma, who proposed a system different from those 
of most other performance contractors (including the Texarkana LSC for the second 
year). These differences are to some extent differences in degree, since most contrac- 
tors seek to modify materials, environment, and incentives, to some extent (but not 
all; some concentrate exclusively on one or two of the three dimensions). 

Use of a Performance Contract 

Many people think of the Texarkana project as merely the contract between the 
district and Dorsett or, later, between the district and Educational Developmental 
Laboratories (EDL). As the list of five items illustrates, a performance contracting 
program can be a much more complex concept. Nonetheless, a contract for educa- 
tional service with the payment based on achievement is the sine qua non of a 
program. 

Two basic types of problems are associated with such contracts: first, their 
legality under state education codes, procurement statutes, union-LEA agreements, 
and similar laws, regulations, and agreements; and second, measurement problems 
associated with the use of tests. The former type of problem has up to now not been 
a major issue in Texarkana . 13 The latter problem has been a very major issue. 

Use of a Management Support Contractor (MSC) 

The management support contractor is an outside organization that assists the 
LEA in a number of ways at each stage of the program. Illustrative tasks are: 
identifying needs, helping to develop a program, preparing RFPs, identifying poten- 
tial contractors, developing contractor-selection criteria and contractor-evaluation 
instruments, negotiating contracts, resolving operating problems that arise, and 
providing cost-effectiveness analyses of program results. Texarkana made extensive 
use of management-support services during the first year and less use during the 
second. 

13 Texarkana made use of legal counsel from the beginning and also had strong community support. 
There is no teachers’ union in Texarkana. 
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Competitive Selection of a Learning System Contractor (LSC) 

In Texarkana and similar projects, the LEA submitted an RFP to a number of 
prospective contractors. The resulting proposals were evaluated and, in the second 
step, the prices and bids were evaluated. Education Turnkey Systems, Inc., the 
Texarkana MSC for 1969-70, strongly believes that this feature not only results in 
educationally superior programs but produces lower costs. 14 



Independent Evaluation and Auditing 

For 1969-70, an evaluation contract was let to the Arkansas Region VIII Educa- 
tion Service Center in nearby Magnolia, Arkansas. Dr. Dean C. Andrew and Dr. 
Lawrence H. Roberts were in charge. Although the evaluators were under contract 
and therefore independent, they were also referred to as internal evaluators. This 
latter usage reflects the fact that their assignment went beyond verification of 
achievement gains and involved them in assessment of processes and educational 
and managerial problems. While it is somewhat unusual to think of an evaluator 
as being both independent and internal, in this program this was the case. 

There was also an independent educational auditor, EPIC Diversified Systems 
of Tucson, Arizona. Its function was to evaluate the evaluation procedures for the 
project and to verify the results reported by the evaluator. The evaluation and auoit 
contracts are contained in Appendixes D and E. We shall have more to say about 
evaluation and audit later because they were strong features of the program. 



THE DORSETT SYSTEM 

Dorsett Educational Systems of Norman, Oklahoma, headed by Loyd G. Dor- 
sett, won a spirited competition for the Texarkana contract. 15 Its response to Texar- 

14 "Competitive Bidding and Turnkey Operations,” Education Turnkey News, Vol. 1, No. 1, April 
1970, pp, 4. 5. 

18 In addition to the contract with the public schools in Texarkana, Arkansas, and Libert y-Kylau, 
Texas, in 1969-70 a Dorsett subsidiary, WOO, had an adult education program in Texarkana funded by 
Model Cities. It involved 43 students. (For a description of the target population, see "Sleeper in Texar- 
kana,” Education Turnkey News, Vol. 1, No. 4, July 1970, p, 4.) This program was concluded during the 
1970-71 school year. The Dorsett group in 1969-70 also had a small Model-Cities-sponsored program for 
nonreading children at Washington Junior High on the Arkansas side. This program was continued in 
1970-71. These Model Cities programs will be further discussed in Sec. II. 





kana’s RFP was judged best in terms of both the substance of the program and the 
price bid. 

The proposal submitted to Texarkana by Dorsett Educational Systems is ex- 
cerpted in Appendix B. Basically, it had three thrusts. Dorsett proposed to replace 
the conventional textbooks and materials with new materials designed around the 
audiovisual equipment that was his Arm’s main product prior to the Texarkana 
project. His educational approach emphasized the use of individualized pro- 
grammed materials and contingency management, Dorsett also stressed environ- 
mental modification —changing the "image” of the classroom and the teacher. The 
RLCs were carpeted and air-conditioned. Air-conditioned classrooms are rare in 
Texarkana despite the region’s hot climate. Dorsett’s teachers were given new titles 
and special blazers to wear. Teachers were hired from a list of applicants for teach- 
ing positions in Texarkana who had not received positions because of a lack of 
opoiiings. All were college graduates and legally certifiable, but not everyone had 
a credential or teaching experience. The aides were recruited in part from the 
Texarkana Titans, a local football team. Instead of school desks or carrels, Dorsett 
constructed office-like desks. In short, every effort was made to differentiate the 
environment surrounding the Dorsett system from the conventional cl as sroom. 

Perhaps the most controversial feature of the Dorsett system was the use of 
extrinsic motivators or monetary rewards for students. The educational theorists 
who designed Dorsett’s system strongly believed in the need to provide disadvan- 
taged students who had become "successful failures” with some new motivating 
force. Thus, access to reinforcement centers, Green Stamps, and radios were given 

for completion of lessons, and a television set was to be awarded to the outstanding 
student. 

Considering the furor that the reward system generated in the educational 
sector, the reaction of some of the people connected with the program, now that it 
is over, is perhaps interesting. Loyd Dorsett told Rand that he now believes extrinsic 
motivators to have been of small value. He believes that his system is interesting 
enough and that the students are interested enough in the "hardware” so that no 
extrinsic motivators are needed. 

A Texarkana official close to the program disagrees on both points. He told Rand 
that he believed the Green Stamps and other rewards were helpful in the beginning 

* Because performance contracting has become identified by some people with the extensive use of 
audiovisual equipment, it is perhaps useful to point out that most other performance contractors are less 
oriented toward teaching machines. 



of the program to capture the student's interest but that the motivating force of the 
rewards declined rapidly. The observer believed that the most important motiva- 
tional techniques, apart from academic success itself, were the traditional practices 
of performing other program functions, and having access to leisure time in the 
reinforcement center. 

The radios and the television set have received much publicity in the press and 
in educational circles. A point that has not been mentioned is that in the con fus ion 
f bat surrounded the test-teaching scandal, many rewards were not awarded. On a 
visit to Texarkana during the summer of 1970, we noticed a number of radios and 
the television set in a storeroom. Texarkana officials told us that the school district 
was not in the business of hunting up students and handing out rewards. Dorsett, 
when we asked him about it, stated that he was no longer involved wl h the Texar- 
kana schools; he had made the radios and television set available and then it had 
ceased to be his business. What the students made of the matter, we could not find 
out. Apparently it was not a matter of sufficient importance for them to come around 
and ask the schools for the rewards due them. 

Teacher bonuses were also much discussed. Apparent!:; some Dorsett stock was 
distributed to Dorsett s employees, but not on the basis of student performance. 
There was also talk of stock and stock option bonuses. The test-teaching scandal so 
depressed Dorsett ’s actual and prospective earnings that such promises were not 
very valuable. Teacher compensation was the same as for regular school employees, 
although overtime pay boosted take-home salaries. 



THE 1969-70 PROGRAM 17 

The program involved 351 students— 219 at the start and 132 later in the year. 
There were 208 male and 143 female students. The racial division was 203 white and 
148 black. Six junior and senior high schools were involved. The students were in 
grades 7 through 12 and most spent two hours in the RLG— one hour on reading and 
one hour on mathematics. The initial group was composed of roughly equal numbers 
of (1) volunteers, (2) students selected by counselors, and (3) students randomly 

"This description is based in part on D, C, Andrew and L. H. Roberts, Final Evaluation Report on 
the Texarkana Dropout Prevention Program, Region VIII Education Service Center, Magnolia, Arkansas, 




selected. The target population was potential dropouts; operationally, this was 
defined as students two or more grade levels behind in reading and mathematics 
tests and having an IQ above the special education level of 75. These tests were the 
Iowa Test of Basic Skills (ITBS) in Arkansas, and Science Research Associates 
Achievement Tests (SRA) in Liberty-Eylau, In fact, a number of students in the 
Dorsett RLCs did not meet the criteria. Of the initial 219 students, 57 had either 
lower IQs or higher reading scores, and for 37 there were no initial test scores. 

Four centers were in trailers and t,.o in refurbished classrooms. As discussed 
previously, considerable effort was expended to m ike them comfortable and attrac- 
tive. The Dorsett M86 Teaching Machine was extensively used. This is a low-cost 
portable audiovisual machine using coordinated records and filmstrips. Other pro- 
grammed materials were also used. For example, at the start of the program it was 
found that more basic materials were needed, and Job Corps texts were obtained. 
Behavioral management techniques such as contingency management and the ex- 
trinsic motivators previously discussed were also a feature of the program. 

The project director was Martin J. Filogamo, an experienced teacher and princi- 
pal in the Texarkana, Arkansas school system. He was hired in August 1970 and had 
an administrative assistant after October 1970. Both were on the LEA payroll. 

Filogamo exercised the cognizance and control functions for the school system. 
Andrew and Roberts state that he viewed his function as being the principal of the 
six learning centers. Filogamo also served as an administrator, making a host of 
decisions about contractual questions and USOE funding matters, as well as pro- 
gram development. Further, he served as host to about 800 visitors, answered over 
700 requests for information, and traveled throughout the United States making 
speeches and attending meetings and conferences on the Texarkana program. 

Dorsett was represented by a resident director, Charles J. Donnelly, who was 
responsible for daily direction and coordination of the centers. Each RLC had a 
manager (teacher) and a teaching assistant (aide). All were carried on the Dorsett 
Educational Systems payroll. 1 8 

Others were involved with the program on a part-time basis. They consisted of 
the management support group, the evaluators, the educational auditors, Dorsett 
consultants, and an in-service cadre. The latter consisted of 20 Texarkana teachers 
who met at regular intervals for instruction in operating the RLC with a view to 
later furnkeying the operation. Each member of the cadre was paid for his time. 

18 This is an atypical feature of the 1969-70 Texarkana program. In most performance contracting 
programs, teachers remain on school district payroll to simplify administration and avoid problems of 
having different salary levels in the same schools. 




COSTS AND PRICES 



The pricing arrangement in the Texarkana-Dorsett contract (see Appendix B) 



Method of Cost Reimbursement 



|; 



l 



$■ 

¥ 



P 
1 = 



I' 

i. 



i 




v 




a. In consideration for services rendered, Dorset! will be compensated 
on the basis of actual student performance. 

b. The student performance differential is determined by subtracting 
the entering grade level achievement in math and reading from the exit 
level. Entry status and exit status are based on the SRA and ITBS tests as 
weighted on a basis to be determined no later than February 1, 1970. This 
procedure will be applied to all assignees except withdrawals, and a small 
number of students, assigned by nonrandom procedures, to be mutually 
agreed by the parties to this contract, for whose learning services Dorsett 
will be reimbursed at the average hourly rate of other students. 

c. Dorsett will be compensated on the basis of obtaining one grade level 
increase per subject area in eighty hours of instructional center study for 
$80.00 or proportionally for each fraction thereof. According to the formula 
stated in the Dorsett Proposal, one grade level increase per subject area in 
60 hours of instruction would cost $106.67 and one grade level increase per 
subject area in 110 hours of instruction would cost $58.18. Both parties agree 
that $58.18 for 110 hours required and $106.67 for 60 hours required repre- 
sent the lower and upper limits of the cost reimbursement formula, with the 
following exception: 

d. Dorsett will grant a 7% discount on the dollar payment per grade- 
level-subject-increase if the average I.Q. of the students who are randomly 
selected is 100 or greater. The procedure for the selection and the termina- 
tion of discount for additional . assignees will be mutually agreed upon no 
later than February 1, 1970. 

e. The parties agree that the price of the $80/80 hour formula will be 
decreased as the operations are extended to additional students beyond the 
initial 200 enrollees to reflect operational efficiency through economies of 
scale, if any. 

The payoff function for each subject was P = $80(HA)/80 hr, where P = pay- 
ment per student, A = grade level advance, and H = hours of instruction Several 
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constraints were placed on this function. The maximum payment for the contract 
could not exceed $135,000. The payment per student per subject could not exceed 
$106,67. If more than 110 hours were required for a grade-level advance the payment 
decreased by $1 for each additional hour (P = 0 if H j 168 for A = 1). 

The payment was also constrained in another way that has not received as 
much attention. Not all students were included in the achievement/price arrange- 
ment. Those students not assigned ''randomly”— and by implication those students 
not included in the target population— were to be paid for on the basis of a flat rate 
per hour of instruction. The flat rate was to be determined by the average payment 
per hour of instruction of those students included in the achievement/ price arrange- 
ment. The contract speaks of a "small number” of students not randomly selected; 
in the event, this number amounted to two-thirds of the students selected. Moreover, 
owing to selection difficulties (such as one school’s pre-test being misscored ) 10 it 
appears that not all the randomly selected students were from the target population. 
Also, some of the target population probably moved, failed to attend for the mini- 
mum number of hours, or did not have both a pre- and post-test. Applying the 
contract rigorously would have meant that the achievement gain scores of a small 
subgroup would determine the payment for all students. 

In fact, however, Texarkana and Dorsett agreed to interpret this clause loosely 
and apply the achievement-payment formula to all students in the target population 
even if they had not been selected randomly. 

In addition to the payment for instruction, Dorsett also sold Texarkana teach- 
ing machines, other equipment, and materials. The profit, if any, on these sales must 
be considered from Dorsett J s point of view when computing the risks involved in the 
contract. 

In sum, to say that a performance contractor is paid on the basis of test scores 
is an oversimplification. More precisely, the cost of instructing some students will 
be reimbursed based on test scores. How many students fall in this category and how 
many in other categories depends on how the target population is defined, how the 
students in the program are selected, how the payment is arranged for students 
outside the target population or outside the "guarantee,” and how many students 
take pre- and post-tests. Also, in addition to the payments for achievement, contrac- 
tors may get other payments, for example, for materials or training or other ser- 
vices. 

The Dorsett price-bid was heavily influenced by his forecast of substantial 
19 Bumstead, op, cit., p. SO. 
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achievement gains, Donnelly, formerly Dorsett’s resident manager, 20 stated that 
with penalties and incentives the payment per achievement year could have varied 
from $59 to $109, Donnelly computed the actual cost to Dorset! at $68 plus home/ 
office overhead of $13 for a total of $81 per achievment-year of gain. He computed 
the Texarkana school system cost as $125 per achievement year, 81 

Very few good cost figures exist. The Dorset! figures used by the evaluator are 
incomplete and perhaps inaccurate. A rough ballpark estimate of what it might cost 
a school district to run a Dorsett-like program can be obtained by examining the 
resources used, as shown in Table 2, Assume that the six teachers each received 
about $6,500 per year and each of the six aides received about $3,000. Also assume 
that the 95 M-86 machines cost $200 each and that the materials, supplies, and 
incentives cost $20,000. Training and consulting might then amount to $3,000. 
Finally, figuring $2,000 per trailer or classroom for remodeling and furnishings 
yields a total of $12,000, On this rough basis, such a program might cost $111,000. 
We hasten to add that these figures do not include supervision cost by either the LEA 
or LSC. Moreover, the figures do not reflect what the program may have cost Dorsett. 
The figures are, at best, what one might expect to pay for a Dorsett-like program 
using cost factors that are typical in the Arkansas area. 

The Texarkana-Dorsett contract remains unsettled as of this report, December 
1971. As will be discussed later, the evaluators decided that no meaningful post-test 
data could be obtained. Dorsett disagrees. He told us in August 1971 that unless 
Texarkana accepted one of his offers to settle the contract by means of purchase of 
material, he was going to institute a legal action promptly. Even if one were to 
accept Dorsett’s gain data, however, only about half the students in the regular 
program (49 or 51 percent, depending on calculation method) were in the target 
population and had pre- and post-test data. The achievement scores for this half of 
the group would determine the compensation for the whole group. 



EVALUATION AND AUDIT 

As evaluator, Texarkana chose the Region VIII Education Service Center in 
Magnolia, Arkansas, about 50 miles away. The center serves schools in an eight- 

*° Donnelly left Dorsett Educational Systems to join Quality Educational Development Inc (QED) 
another prominent performance contractor.- 

21 Economics of Performance Contracting.” Education Turnkey News, Vol. 1, No. 9-10, December- 
January 1970, p. 7. 



Table 2 

PROGRAM AMD RESOURCE INFORMATION FOR 1969=70 



Charact eristics of students , . ,*• * Grades 7^12 ; educationally disadvantaged 

(at least 2 years below grade level); IQ 
at least 75 



Program scope 

Number of students. 

Class time 

Class size. ........ 



350, reading and math 
1 period math, 1 period reading 
20 students per classroom area 



Facilities 

Space....... 4 trailers, each 900 sq ft; 2 classrooms, 

each 1000 sq ft 

Utilization... , , 100 percent 

Furnishings,....,, Desks, carpet, air conditioning 



Staffing 

Certified teachers 

Special teachers 

Paraprofessionais , , , f , 

Other personnel,. 

Equipment * * . , 

Materials 

Pre-service training 

In-service training 

Incentives 

Students ,,,,,, 



Teachers 



1 per center 
0 

1 per center 
Project manager 

Dorsett M-86 Teaching Machines 

Filmstrips, records, programmed texts 

8 days per teacher 

No formal training 

Green Stamps, transistor radios, 1 TV, 
some popular records (and player), games, 
puzzles, popular magazines, free time 
Dorsett stock bonus and options 




county area and is partly supported by a Title III grant under the Elementary and 
Secondary Education Act of 1965. The center offers training services and program 
planning, implementing, and evaluation services. It has a professional staff of 11. 
The evaluators not only had appropriate professional backgrounds, but the advan- 
tage of knowing Texarkana schools and being geographically close so that they could 
interact with the program- frequently and on short notice. 

More important, the Region VIII center was designated the Internal (as well as 
Independent)-' Evaluator and provided managerial feedback during the program. 
The evaluators used a "contact report” form filled out on each visit to record events 
and problems that arose as they happened. These reports provided a mechanism for 
identifying problems for program management and suggesting resolutions. This 
mechanism was extremely important in the test-teaching episode, 

Texarkana chose EPIC Diversified Systems Corporation of Tucson, Arizona, to 
be the educational auditor. EPIC offers various evaluation, need-assessment, plan- 
ning, training, and audit services. EPIC is an outgrowth of the EPIC Evaluation 
Center, also a Title III project. The Region VIII center uses for its evaluation activi- 
ties the EPIC Scheme of Evaluation developed by the EPIC Evaluation Center. 

The project budgeted $11,500 for the internal evaluation and $5,400 for the 
audit. The difficulties in settling the contract with Dorsett led to some additional 
evaluation and auditing work. 

The Evaluator’s Task 

The Final Evaluation Report 22 prepared by Dr. Dean C. Andrew and Dr. Law- 
rence H. Roberts of the Region VIII center in Magnolia is a model of organization, 
clarity, statistical sophistication, professional expertise, and good sense. There was 
a serious lack of coordination, however, between the evaluation design and the 
initial planning for the contract and program. As the Final Evaluation Report 
explains: 

The operation of the Texarkana Dropout Prevention Program officially 
started on November 3, 1969, with four of the rapid learning centers start- 
ing. The other two started early in January, 1970. The internal evaluators 
were employed after the start of the project and held their initial meeting 
with the project director on November 13, 1969. The internal evaluators 



22 Andrew and Roberts, op. cit. 



were not involved in the initial planning; and thus, procedures, information, 
and design had to be developed "after the fact.” To further complicate the 
evaluation process, the internal evaluatoi,. did not have access until Janu- 
ary 13, 1370, to the original proposal which had been submitted for the Title 
\ III, ESEA grant. Due to this fact, the internal evaluators took the Dorsett 
Company’s proposal and wrote behavioral objectives based on what the 
company said they would accomplish. The evaluation design was based on 
these objectives. After the objectives were written, the official contract with 
Dorsett was finalized. The contract did not require Dorsett to do some of the 
tasks that were written in their proposal, such as development of study 
habits, improvement of speech, improvement of grooming practices, etc. In 
a conference involving the internal evaluators, the project director, and 
representatives of the U.S. Office of Education held on January 18, 1970, the 
project director was requested by the representatives of the U.S. Office to 
revise the original proposal to more correctly reflect what was being done 
in the program. This revised proposal was submitted in late February, some 
four months after the start of the project and three months after the evalua- 
tion program was implemented. The revised proposal contained some per- 
formance objectives which were not entirely consistent with the evaluation 
design objectives. In a telephone conversation between Mr. Lewis Walker, 
of the U.S. Office of Education and the project director, it was agreed that 
the evaluation objectives be used as the objectives of the current (1969-70) 
operational program, and that the objectives in the revised proposal be used 
as the starting base for the development of the evaluation design for the 
1970-71 continuation grant. 23 

In snort, in the first year of the Texarkana project the program design and 
evaluation design were out of phase. The late start of the evaluation made it difficult 
to gather pre-test data in time for the first exit test. 

A second problem that Andrew and Roberts faced was: Should the evaluator 
evaluate the contract or the program? As Andrew and Roberts note, the goals that 
they believed the program were addressing turned out to be different from the tasks 
that Dorsett contracted to perform. As it worked out, Andrew and Roberts took on 
both tasks; they attempted to determine the contribution of the RLCs to the broader 
objectives of the Texarkana schools. 

23 Ibid., pp. 13-14. (Italics in original.) 
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In part, the problem was due to a confusion in roles. The auditor, EPIC, who also 
produced an excellent report, had been given the task of "providing information for 
decisionmaking" (see Appendix E), Possibly the intention was that the evaluator 
would focus on what the contractor was to receive in the way of payment and the 
auditor was to concentrate on whether the RLCs contributed to Texarkana’s pro- 
gram, which seems the reverse of what one would expect. However, the management 
support group, Education Turnkey Systems, was also responsible for cost-effective- 
ness analyses and general decisionmaking guidance. The roles and functions of the 
three groups, and who had responsibility for a broad program assessment, were not 
obvious during the first year and in fact led to meetings and discussions about 
respective roles, functions, and responsibilities. 

Determining Achievement Gains 



After a thorough and sophisticated analysis, the final report concludes that 
Dorsett had exposed so many test questions that no statistically valid achievement 
gains could be determined. No test sources were published in the Final Evaluation 
Report, 

The Interim Report and the continuation grant application sent to USOE did 
contain scores based on the February, March, and April exit tests. These scons were 
given wide publicity as "miracles” and attracted much attention. The Final Report 
disregards these scores but because of the attention they received we will briefly 
review them. 

There were initially about 219 students in the learning centers, with 351 event- 
ually participating by program end. There were four exit or post-tests given during 
the program, as follows: February 2, 57 students; March 2, 59 students; April 6, 96 
students; and May 22, 285 students. The evaluation design set up a control group 
that was equated with the experimental group (for each test separately) on pre-test 
vocabulary, reading comprehension, and arithmetic scores; and on race, sex, grade 
level, and school. There was no separate control on intelligence, because of lack of 
scores for the controls. For the group tha; took the February test, the experimental 
group showed a significant gain over the control group only in vocabulary (for the 
experimentals, the percentages gaining at least one grade level were 63, 37, and 33, 
respectively, for vocabulary, reading comprehension, and arithmetic). The March 
test group showed a significant gain over controls in both vocabulary and reading 
(with percentages gaining at least one grade level of 49, 47, and 39), and the April 
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test group showed no significant gains over controls. Looking at the multiple testing 
effect, for students taking both the February and March tests because they failed 
the February test, there was a mean decrease on vocabulary for experimentals and 
an increase for controls. There was no mention of significance levels. On scores 
alone, the achievement gains are equivocal but there was also test-teaching, accord- 
ing to the evaluator. 

The Final Evaluation Report includes a large section on the number and per- 
centage of items exposed for each form and grade level of each test. There is also 
an informal chronology of when a given type of item was put in the RLC materials. 
The evaluators concluded that the vocabulary was 100 percent exposed on the ITBS 
and about 85 percent on the SRA, and these items were introduced in concentrated 
lessons before the first (February) exit test. Before the March test, significant expo- 
sure was added on reading comprehension and arithmetic items, in the form of 
review lessons along with priority lists for the center directors (giving the lessons 
to review two weeks before the exit test). Since the last priority list was issued just 
after the April test, the evaluators decided that all tests were taught to varying 
degrees. The evaluators obtained a formula from Educational Testing Service of 
Princeton, New Jersey, from which they calculated the number of exposed items 
required to lower subtest reliability below 80 percent (the level chosen for usability 
for payment, since that was the overall test reliability level). No vocabulary items 
and only half of the arithmetic items on the ITBS met the ETS criterion. On the SRA 
tests, no reading, 25 percent of the vocabulary, and 50 percent of the arithmetic met 
the criterion. 

Since payment was to be based on the average of vocabulary and reading 
together and on the total of arithmetic, the evaluators concluded that r one of the 
exit tests was usable for payment purposes. The actual scores will remain unavaila- 
ble until contract payment is legally settled with Dorsett. 24 It is interesting to note 
that Loyd Dorsett does not consider reliability level in the fact sheet he circulated, 
but merely presents the percentage of exposed items over total items (and his 3 
percent estimate is clearly too low if the evaluator is right about the number of 
questions exposed). Dorsett’s position was set out in two letters reproduced in the 
Final Evaluation Report, 

As of December 1971, the contract was still unsettled, with no apparent action being taken. 
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Determining Program Effectiveness 



Andrew and Roberts consider a number of different program-effectiveness and 
product criteria. 

1. The students were to respond positively to the rapid learning center pro- 
gram as indicated by: (1) a feedback questionnaire, (2) decrease in school absentee- 
ism, (8) decrease in dropout frequency, and (4) improving grades in other classes. 

Due to the many entrance and exit points during the year for the RLC program, 
only 67 students took both the entrance and exit questionnaire. There is a slight 
positive attitude increase, but it is not significant. What is interesting is that most 
of the entrance responses are already pretty positive, which may be because the first 
questionnaire administration didn’t occur until after 3 months of instruction. Or the 
questionnaire might be invalid, since the "entrance” responses seem untypically 
positive for the population. 

The shifting of the RLC population made it difficult to find equated controls, but 
for one sample of 33 RLCs and controls the RLCs had a slightly but not significantly 
higher rate of absenteeism. So the program definitely didn’t lower absences. 

The decrease in dropouts was very dramatic, as discussed below. 

Again, only a sample of between 25 and 32 students was available, for each of 
four subjects in two semesters (the first of the RLC program and the semester 
preceding the program). By inspection, the RLC English grades improved a bit while 
the controls declined, but the grades in the other three subjects declined for both 
groups. The English result is encouraging, but needs control for teacher knowledge 
and expectations. 

2. The student in the RLC program was to display an increased vocabulary, 
reading comprehension, and arithmetic knowledge as indicated by scores on the 
ITBS and SRA achievement tests. 

However, there were no valid scores due to teaching for the test, as noted above. 

3. The initial evaluation design states that: "The project director will display 
knowledge of the feasibility of a rapid learning center program for all students in 
the school system as indicated by: (1) feasible cost, (2) available space, (3) appropri- 
ateness of RLC material, (4) acceptance by faculty and parents, and (5) permanency 
of student achievement gains.” 
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The cost comparison between the RLC programs and the regular school’s cost 
for producing a one-grade level increase in achievement was not made due to the 
invalid post-test scores for the RLC program. Further, Dorsett noted that the cost 
data provided early in July 1970 had a number of omissions. A final cost of $146,357 
is given, which works out to $542 per student, and includes one-time start-up costs. 
However, only about $140,000 of the quoted program budget of $250,000 seems 
allocated to Dorsett. The evaluator was under strong time pressure, for no attempt 
was made to compute the average operating cost for the regular schools. 

The space study wasn’t conducted due to some buildings being phased out under 
a total desegregation plan. It was correctly estimated that only the mobile units and 
some refurbished classrooms would be available for the following year’s turnkey 
learning centers; also, the curriculum wasn’t fully rated due to the project director’s 
lack of time. 

All teachers in the target schools were given a feedback questionnaire on the 
RLC program. Of the 220 that responded, all had heard of the program and 70 
percent had visited an RLC center. Further, most teachers felt that the RLC materi- 
als would be effective in their classrooms, especially with a teacher aide. Thus the 
climate for change was termed good. Finally, for the 73 teachers that had RLC 
students in their classrooms, the majority felt that school attitude and class discus- 
sion had improved, while grades, study habits, class attendance, and appearance had 
not. 



A questionnaire on the program was also given to all parents of RLC students, 
with 118 responding (about half). Almost 97 percent had heard of the program, but 
only 5 percent had visited a center. Even though only a little more than half noticed 
changes in school behavior, about 92 percent wanted the program in the regular 
school and thought it would help their children get ahead and graduate from school. 
This result plus the teachers’ favorable reaction to the programmed materials seems 
a major influence in the decision made later to purchase the Dorsett machines and 
programs for the turnkey centers. 



Retention tests were eliminated when USOE revised the original proposal. 

4. The students in the rapid learning center program were to demonstrate 
increased application of pronunciation skills as indicated by scores on the Photo 
Articulation Test. 25 

The pre- and post-tests were administered to individuals and scored by certified 



Objert^s 4 to 7 were 



not included in the Dorsett-Texarkana contract 



ERIC 
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speech therapists. For 37 RLC students, there was no significant reduction in the 
average errors for the group, but the initial average of 3,5 errors seems small to 
begin with. 

5. The students were to display knowledge of the world of work by choosing at 
least one employment goal that is realistic and achievable as judged by the student’s 
counselor. 

Unspecified time pressure prevented all but 38 of the goals questionnaires given 
to all RLC students (in May) being rated by counselors and returned to the evaluator, 
1 he 38 show a strong realistic rating, but that doesn’t seem very meaningful without 
a pre-test. 

6. The students in the rapid learning center program were to display an in- 
creased knowledge of study skills as indicated by scores on the SRA Study Habits 
Checklist, 

126 RLC students took both the pre- and post-tests (orally administered to mini- 
mize reading difficulty) but showed no significant gain. However, there should have 
been controls, since they might have shown a decrease. 

7. The students were to demonstrate an increased application of good grooming 
as indicated by an appearance checklist. 

Results were unavailable, since the teachers did not have time to do the ratings. 

In short, the achievement test data were unusable and most indicators gave 
unclear signals about the impact of the program. There was, however, one dramatic 
exception— the decrease in the dropout rate, the ultimate goal of the program. 

For 351 Rapid Learning Center students, the dropout rate for the eight months 
of the contract was 6.8 percent while that for 235 equated controls was 17,9 percent. 
Further, the RLC dropout rate was also noticeably below the overall dropout rate 
of 8,3 percent for the 4340 students enrolled in the six target schools. Finally, since 
there were three methods of selection resulting in many students nwc meeting the 
target-population criterion of two grade levels below and IQ of at least 75, the 
dropout rates of RLC students meeting and not meeting this entry criterion were 
compared. For 181 students meeting the entry criteria, the dropout rate was only 
1,7 percent, while for 170 students not meeting criteria the rate was 12,4 percent. 
Hence, the contract program was particularly effective for those students that met 
the program criteria, i.e., the population for which it was designed. 



Educational Process Difficulties 



A host of operating difficulties, such as lack of records, confused schedules, and 
similar problems plagued the program. The evaluators chronicled a large number 
of these and then explained them as follows: 

Some were results of lack of planning; others were due to lack of guidelines; 
some were due to unclear understanding of roles and functions; some were 
due to slow negotiations; some ... to a reduction in funds; and others were 
unanticipated problems that occur when any new developmental program 
is initiated . 28 

Any innovative program will encounter start-up problems. Funding cutbacks 
also create problems. We would emphasize two other causes, however. First, the 
program involved a large number of persons and organizations. There were the 
Texarkana project director and his office, Dorsett and his firm, the management 
support contractor, the evaluator, the auditor, the Model Cities group, and USOE, 
the sponsoring agency. How the roles and functions of these various groups were to 
be distinguished and how their activities were to mesh should have received more 
planning attention. In the pressure to get the program under way, these matters 
were left to work themselves out with resulting confusion. 

Second, the evaluation was not coordinated with product planning and contract 
negotiation. The two aspects were out of phase and again produced confusion. 

It should be emphasized that these phenomena were not peculiar to Texarkana. 
We have encountered them in most programs. 

Test-Teaching 

It was understood from the start of the program that Dorsett had analyzed the 
ITBS and the SRA in designing his materials. It was also understood that he would 
teach to the test” in the sense of teaching the skills tested by those instruments. 
By common consent this would, for example, permit teaching a student how to divide 
a common fraction by another common fraction, if such a skill were to be tested by 
the ITBS or the SRA. However, it was assumed that the numbers in an exercise 
example would differ from the numbers in test questions. 



89 Andrew and Roberts, pp, 33-34, 



In reading, it was understood that Dorsett proposed to teach a 3000-word 
vocabulary containing the words encountered on the ITBS and SKA. However, 
drilling on words actually used in the specific instruments was regarded as beyond 
acceptable limits. 

The strategy of reading test paragraphs was regarded as being within admissi- 
ble standards for teaching reading comprehension. Putting into daily exercises the 
actual paragraphs used on the test instruments was regarded as inadmissible. 

It should be noted that despite these general understandings, no specific guide- 
lines for distinguishing inadmissible from admissible practices were furnished Dor- 
sett. After the issue arose during the May exit testing, the Educational Testing 
Service was called in as a consultant to the evaluator and provided the guidelines 
reproduced in Table 3. These guidelines have been made a part of several subsequent 
contracts in Texarkana and elsewhere, 

Dorsett admitted that a number of test questions had been exposed. The issue 
then became the extent to which the achievement data were polluted. Dorsett’s basic 
procedure has been to take the gains (which he believes averaged 1.5 grade levels 
in both reading and mathematics) and discount them for the number of exposed test 
items. He concluded that the reduction in gains was less than 3 percent. 27 The 
evaluators’ procedure, based on suggestions from Educational Testing Service, was 
to determine the number of items that could be exposed and still yield a statistical 
reliability of .80 for the given group of test items. Using this procedure they decided 
that all the May scores were invalid. 

The Texarkana School Board was placed in the position of adjudicating the 
dispute. Dorsett threatened legal action if the evaluators’ position were accepted. On 
the other hand, the Board was hard put to ignore an independent evaluation by 
competent and respected experts that it had hired. The issue of how much Texar- 
kana owes Dorsett remains unresolved to this day. 

There are two morals to this story. First, expect the expected. There was a great 
likelihood that any contractor might step across the line from admissible to inad- 
missible teaching-to-the-test. At a minimum, rules such as those given Dorsett in the 
spring should have been furnished the previous fall. 

The second moral is: have a fail-back position. It would have been much easier 
for Texarkana had some procedure for resolving the contractual obligations been 
written into the contract. It is not merely a matter of test-teaching; strikes, unusu- 
ally bad weather, logistics problems, and a host of other possible difficulties can 

27 Catalog of Programs, Dorsett Educational Systems, Inc., Norman, Oklahoma, 1971, p. 3. 



Table 3 

EDUCATIONAL TESTING SERVICE GUIDELINES 



A test item and an instructional exercise are to be considered the same. if 

(1) Their wording is identical in all respects despite a change in format. 

Example i A, Which of these is a way to find the circumference in inches of a 
circle with a 6-inch diameter? 



(1) 3x3.14 

(2) 6x3,14 



(3) 3x3x3.14 

(4) 2x6x3,14 



B, Which of these is a way to find the circumference in inches of a 
circle with a 6- inch diameter? 

(1) 3x3.14 (2) 6x3.14 (3) 3x3x3,14 

(4) 2x6x3.14 

(Note the change in mvcmgmeni of options) 

(2) The wording of the stem and the wording of the correct option are identical; despite 
the fact that other options have been changed. 



Examples A, 



Which of these is a way to find the circumference in inches of a 
circle with a 6-inch diameter? 



(1) 3x3,14 

(2) 6x3.14 



(3) 3x3x3.14 

(4) 2x6x3.14 



B, 



Which of these is a way to find the eireumferenea in inches of a 
circle with a 6-irtch diameter ? 



(1) 3,14x3 

(2) 6x3.14 



(3) 3x2.14 

(4) 2x6x2.1416 



(3) Tlie correct option Is 'Identical and thA main sense of the stem has been retained 
despite a minor change ip^wording. 



A, 



Which of these is a way to find the circumference in inches of a 
circle with a 6— inch diameter? 



(1) 3x3.14 

(2) 6x3.14 



O) 3x3x3.14 
(4) 2x6x3,14 



B. 



The number of inches In the circumference of a circle with a diameter 
of 6 inches Is 



(1) 6x3,14 

(2) 3x3.14 



(3) 3x3x3.14 

(4) 6x6x3,14 

U:. 
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Table 3 — con ti nuod 



(4) The main sense of the whole item has been retained despite the fact that it has been 
restated in the negative. 

Example ; A, Which of these is a way to find tha circumference in inches of a circle 
with a 6-inch diameter? 



(1) 3x3.14 

(2) 6x3,14 



(3) 3x3x3,14 

(4) 2x6x3,14 



B, The number of inches in the circumference of a circle with a diameter 
of 6 inches is not 



CD 6x3.1416 
22 

(2) 6x -y 



(3) 3x3x3,14 

(4) 2x3x ~ 



(5) The main sense of the s tem has been retained despite a minor change in wording; 
the correct option is Identical; but one or more inoorreot options h,. been 
changed or omitted. 

Example; A, Which of these i # s a way to find the circumference in inches 
of a circle with a 6-inch diameter? 



(1) 3x3.14 

(2) 6x3.14 



(3) 3x3x3,14 

(4) 2x6x3.14 



B, The number of inches in the eiremnference of a circle having a 6- in ch 
diameter can be found by which one of these? 



(1) 3x 



(2) 6x3,14 



(3) 3x3x3,14 



(6) The item has been changed from a multiple-choice to a true-false item by retaining 

the stem of the multiple-choice item and incorporating in the stem one of the options 
(correct or Incorrect), 

Example. A, Which of these is a way to find the circumference in inches of a 
circle with a 6-inch diameter? 



(1) 3x3,14 

(2) 6x3.14 



(3) 3x3x3,14 

(4) 2x6x3. ? 4 



The number of inches in the circumference of a circle with a 6- inch 
diameter is 3x3x3,14 



Trui 




C. A circle with a 6- inch diameter has a circumference of 6x3*14 inches. 
True ( ) i '» lse 




pp, 58-59. 
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SOURCE; Andrew and Roberta, op. cit . , 



make it impossible to compute achievement gains as originally intended. A contin- 
gency plan would seem to be a program essential. 28 Unfortunately, we know of no 
performance contract with such a plan, and we know of several in which, ejc post 
facto, the parties would have liker to have one. 



MODEL CITIES SUPPORT 

The involvement of the Model Cities Demonstration Agency in the performance 
contracting program has been vital and has been too little discussed. Not to take 
credit away from the Texarkana schools, it is still fair to say that Model Cities has 
been available at some crucial times. 

The role of Model Cities in getting the program going has been noted. The next 
helping hand came in November 1969, when Donnelly and Filogamo discovered that 
Washington Junior High had a great many nonreaders and too many students for 
one center. By January, Model Cities had financed and helped plan for a Reading 
Clinic for the nonreaders, and a second RLC was put in operation. The Reading 
Clinic brought to fruition one of the three "Related and Ancillary Programs” men- 
tioned in the 1969-70 RFP, and was crucial in helping Dorsett handle the demanding 
all-black Washington Junior High. 

The Reading Clinic contract, signet with Dorsett in January 1970, was a sub- 
contract since the Texarkana schools hau already contracted with Model Cities for 
the necessary money. It was also a performance contract of sorts. If the contractor 
did not achieve an average gain of at least 1.25 grade levels, both parties agreed to 
negotiate cost payment, with the Title VIII formula used as a guideline. Similarly, 
a bonus was possible for gains above 3.75 grade levels. About $25,800 wnuld be paid 
in gradually decreasingUxed amounts, approximating an instructional cost per hour 
of $1.88. As with the Title VIII contract, equipment was purchased separately, in 
advance. 

By February 1970, 43 Washington Junior High students in grades 7, 8, and 9 
were selected by the principal and the counselor, and classes were begun. The 
instruction was done by two young college graduates in a mobile trailer, primarily 
using the Evans reading materials (developed under EVCQ, a subsidiary of Dorsett), 
The students were given a large number of tests, including: a pre and post of the Job 

88 This point is also made in Education Turnkey News, Vol. 1, No. 3, June 1970, p, 5. 



Corps test, pre and post of the ITBS, the Botel Word Recognition and Comprehension 
tests, the oral Slosson IQ test, and the Slosson drawing test to detect brain damage. 
The average IQ was about 70, and 23 students scored below the first grade level on 
the Botel. It was clearly a challenging group to try to motivate and educate in just 
four months. 

The program ended about May 30th, with 40 of the original students still there 
(but anly 5 had been able to finish the Evans course). On the Job Corps tests, the 
average increase was 1.3 grade levels, with 16 who gained 1.5 or more and four who 
gained from 1.0 to 1.5 (average pre was 1.9, average post 3.2). There were only 10 
post-test scores on the ITBS, but the pre and post averages (5.0 and 5.2) were not at 
all comparable to the Job Corps test. The program seems successful, and the evalua- 
tion report (by a local reading program coordinator) indicates respectable cognitive 
gains. The program was funded again in 1970-71 by Model Cities. 

Just after the Clinic started February, it was obvious that management 
support would be needed to plan the Continuation Grant Proposal and to do a 
turnkey analysis. Model Cities then provided the money to execute a nevv contract 
with Blaschke in February 1970. Blaschke provided valuable inputs to the Phase III 
planning, but did not have enough data from the first year to do a suitable turnkey 
feasibility analysis. 

The second of the "Related and Ancillary” programs to be established was 
called "Operation Second Chance.” Its stated purpose was to provide the instruction 
necessary (including reading and mathematics) to allow Model Neighborhood resi- 
dents to take and pass the General Equivalency Degree examination for high school 
equivalency. It started in March 1970 as a performance contract between Dorsett 
and the city’s nonprofit Manpower Development Corporation to give at least 95 
Model Neighborhood residents GED-ielevant instruction in two mobile trailers 
located near Model Cities Neighborhood Centers. The Citizens’ Education Commit- 
tee of Model Cities and Donnelly helped plan it, adapting many of the Title VIII 
provisions directly. The contract had a maximum payment of $37,000, using the 
$80/80 hr formula of the Title VIII contract for reading and math, and $200 and 
$100 bonuses for passing the GED (since the other contributing subject could not be 
measured directly). The contract ran until November 1970, and payments were 
scheduled to be made at a rate of 10 percent per month up to 85 percent of the 
contracted amount. Two dates were also specified when the rate could be negotiated 
up or down, based (in an unspecified manner) on test results. Also, 32 M86 Dorsett 
machines were purchased initially at $200 each. 



The program started out with the difficult goal of providing basic education for 
adults, and it had many operational problems as well. The trailers were to be placed 
in the two neighborhoods that recruited (competitively) the most students. Even 
though about 170 were pre-tested, only a little more than 40 had signed up by the 
middle of the summer, and new recruiting was necessary. One of the lab directors 
said that community interest in the program was quite high. But there were many 
practical problems, such as mothers needing day care for children and people having 
to spend too many hours at the center. Many were also discouraged by the three 
hours necessary for the pre-test and never showed up after that. 

Turning to results, the lab director said that only 6 had earned GEDs, but 
among other problems there were at least 15 students that were ready for the GED 
exam but were too young to take the test. They were largely high school dropouts, 
notably girls. Later, the monitor said that of the first 21 post-tested, there was an 
average reading gain of 1.5 grade levels and about 1 grade level in math. For the 
40 recently tested, the scores were "a little fewer.” With these difficulties, contract 
payment provisions were renegotiated, the result being that Dorset! earned a re- 
spectable $14,000 on achievement, but only about $20,000 over all because so few 
passed the GED exam. Over all, then, Model Cities provided very necessary support 
to the planning and operation of the 1969-70 Title VIII program. They also funded 
several key expansions of the reduced Title VIII program, and thus kept alive the 
larger focus of an overall manpower education program as the setting for dropout 
prevention and academic remediation efforts. 



CONCLUSIONS 

Looking back on Texarkana in 1969-70 from the vantage point of the fall of 1971 
and the perspective of a second year of performance contracting experience, we are 
struck by the extent to which the issues and problems that arose in Texarkana that 
year are still relevant. 

The first in importance is testing. Not only are tests difficult to administer, but 
the need to define inadmissible test-teaching and to set up safeguards is essential. 

A second problem is how to settle achievement payments when the conditions 
assumed in the contract do not apply. In the Texarkana case in 1969-70, the problem 
was that the evaluator decided that the achievement test data were statistically 
invalid. Other programs have encountered other problems; for example, classrooms 



were not ready in time to provide the contractor with the minimum number of days 
of instruction called for in the contract. The need for a fail-back system for contract 
settlement would seem to be a clear implication of the Texarkana experi ence, but 
it has yet to become a feature of performance contracting. 

A third implication is that performance contracting creates a significant work- 
load for the school district as well as for the contractor. Performance contracting, 
Texarkana shows, is not a matter of calling a contractor in and then sitting back 
to await evaluation reports. Considerable district inputs are required for a useful 
program. Even in Texarkana, where local managerial and analytical resources were 
augmented by contracts for management support, evaluation, and auditing, the 
program placed a considerable burden on the Texarkana school district manage- 
ment and required an office with several talented officials to oversee and direct the 
program. 

The first year of performance contracting in Texarkana also indicated that 
questions of role definition will arise, especially when multiple contracts are in- 
volved. This problem came up in several forms. The specific responsibilities of the 
internal evaluator, the auditor, and the management support group made up one 
problem. Another was assigning the respective supply, procurement, and other 
logistics responsibilities of the school district and the contractor. Another was am- 
biguity in the responsibilities of the RLC personnel to the contractor and to the 
school district . 89 

A final implication of the program is that change is endemic to any performance 
contract. One of the admirable features of the Texarkana program is that it was 
planned for five years. Even if it had not been deemed necessary to terminate 
Dorsett’s involvement, it would have been difficult to achieve the ambitious goals of 
the program in a year or two. In fact, it was possible to incorporate many lessons 
learned the first year into the second year of performance contracting in Texarkana. 

A corollary to the flexibility entailed by a multiyear program is the desirability 
of having a source of supplementary funding, such as the Model Cities support, if 
one is pursuing such broad goals as those Texarkana outlined in its first dropout- 
prevention proposal. 

29 To cite a trivial example, for a time RLC teachers enjoyed soft drinks in the RLCs, while regular 
teachers were forbidden to take drinks into their classrooms. The RLC personnel were instructed to obey 
school rules. 





III. THE TITLE VIII DROPOUT PREVENTION PROGRAM. 



1970-71 



THE PROGRAM DESIGN 

Despite the difficulties attendant upon the test-teaching scandal, Texarkana not 
only continued the Dropout Prevention Program but expanded it and made some 
substantial changes. The major changes were a heightened emphasis on the dropout 
prevention goal and on the goal of introducing new educational processes and tech- 
nology into the regular school program. This latter goal was strongly reflected in 
Texarkana’s outline of its implementation strategy, reproduced in Fig, 2, The Drop- 
out Prevention Program was to be divided into four stages spread over seven phases 
(each a school year). The stages were: Development (of curriculum), Experimenta- 
tion (testing of curriculum in special centers), Demonstration (special turnkey class- 
rooms), and Adoption (introduction of new curriculum into all classrooms). Eight 
subjects were scheduled through the four stages. The subjects scheduled for develop- 
ment and experimentation during 1970-71 were vocational education and consumer 
education, but they were later ignored because management was overloaded. Read- 
ing and mathematics were scheduled for turnkey demonstration. 

This broad strategy is reflected in the organization of the project, charted in Fig. 
3. The project was divided into four components: learning center, turnkey, counsel- 
ing and guidance, and curriculum and instructional. The learning center component 
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Fig. 3 — 1970-71 Project organization 



comprised the LSC’s activities. The turnkey component was to introduce learning 
center technology and processes into regular classrooms. The counseling and guid- 
ance component was responsible for working with potential dropouts to develop 
their self-esteem and aspirations. The curriculum and instructional component was 
responsible for curriculum redesign in English, Mathematics, and Vocational Edu- 
cation, and for teacher training and development in individualized instruction and 
contingency management. 

The turnkey component evolved naturally from the 1969-70 program design. 
The addition of the counseling and guidance component reflected a decision to attack 
the problem of dropouts directly by working with them on career and personal plans 
and not to limit the attack to the indirect technique of improving academic skills. 
The final component reflected a desire to use the program to design a workable 
curriculum and to generally upgrade teaching skills. 

A second change in the program between 1969-70 and 1970-71 was a reorganiza- 
tion of functions. The management support function, even though it occupies a 
separate box in Fig. 8, was in practice part of the internal evaluation function. Both 
the internal evaluation and most of the management support were provided during 
1970-71 by the Region VIII center in Magnolia. The involvement of Education 
Turnkey Systems, Inc., whose president had been a prime mover in the project, 
effectively ended with the preparation of the continuation grant application to 
USOE in the early spring of 1970 and the new RFP in June 1970. Though not shown 
in Fig, 8, EPIC remained the program’s educational auditor. 

Development of a planning, programming, and budgeting system (PP T jS) was 
also added to the 1970-71 program. A contract with Educational Consultants of 
Athens, Georgia, was awarded for management support for this effort (see Appendix 
I), Also, each component had an official responsible for it. Overall responsibility 
remained with the Project Director, Martin J. Filogamo. 

A third noteworthy change in the program was that the objectives and payment 
provisions were no longer linked solely to norm-referenced test results. The RFP for 
the 1970-71 program stated the long-range goals of the program to be: 
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1. To significantly reduce the percentage of dropouts in the Texarkana and 
Liberty-Eylau school districts, 

2. To increase academic achievement and skill development of students who 
are educationally deficient. 

3. To increase the cost-effectiveness of the instructional program in the Tex- 
arkana and Liberty-Eylau school districts. 







The EFP then stated that the contractor for the learning system component 
must guarantee that there will not be more than a 5-percent rate of dropouts from 
the program. The RFP also stated that standardized norm-referenced tests "if used 
alone” were inadequate, and that part of the payment to the contractor must be 
based on criterion-referenced tests. 

The learning center component and the turnkey classrooms were to be embed- 
ded in the larger and more comprehensive effort. However, since this study is 
directly concerned with performance contracting, attention will be limited to the 
turnkey and learning system components, in that order. Both the curriculum and 
the counseling components languished because of pressing problems elsewhere, such 
as managing a new RLC program, keeping turnkey going, and managing the mas- 
sive testing effor:. 



TURNKEY COMPONENT 

The term turnkey comes from the construction industry, 00 When a utility plant 
or some other complex unit of construction is to be built, construction contractors 
will sometimes take complete responsibility for it from the start until the entire unit 
is functioning. They will design, construct, and equip the unit and insure that it is 
operating successfully before "turning the key” over to the buyer. The analogous 
operation in education is the development of a learning system by a private firm that 
tests the system in actual operation in a school district, gets it performing well, and 
then presumably turns it over to the LEA for in-house use. But transferring the 
technology and process-practices for a learning system is rather different from 
handing over the key to a construction unit, as Texarkana showed. 

Plans for Turnkey 

Even though the turnkey concept was to be the keystone of the original Texar- 
kana dropout prevention program,, the initial program documents said relatively 
little about it. In the Addendum (July 1, 1969) to its original proposal to USOE, 
reflecting accommodation to a severe budget cut, only two operational objectives 

J0 For more on the turnkey concept, see Education Turnkey Syst ems, Inc,, Performance Contracting 
in Education, Research Press, Champaign, Illinois, 1970. 



bore on turnkey. The first was the training of 20 junior and senior high school 
teachers, working as part-time "consultants” to the contractor, to use the contrac- 
tor’s programs in their classrooms. This training was to be on the job rather than 
in formal workshops. In addition, turnkey might draw from the estimated 15-20 
paraprofessionals who were to be employed by the contractor directly in the learn- 
ing center vonly 5 aides were actually used in 1970-71). The second relevant objective 
was to determine the specific costs and instructional time required to produce grade 
levels of achievement for students with a given profile. This was to have provided 
data for planning the turnkey phase and was stated as being the most important 
objective, requiring the most careful planning during Phase I. 

Little time was available for planning turnkey operations during the summer 
of 1969, which was filled with an RFF, proposal evaluation, hiring a project 
manager, and selecting a contractor. During the rest of 1969, attention was neces- 
sarily focused on getting the learning centers going. In addition, since no require- 
ment was specified for Dorsett to submit the cost reports necessary to allow the 
evaluator to determine "cost per achievement gain,” the monthly operational and 
cost reports never came in. Also, in the Final Evaluation Report for 1969-70, the 
evaluator noted that Dorsett had four working sessions with a "cadre” of 20 teach- 
ers, including curriculum evaluation, but the first meeting was not until January 
20, 1970. Planned demonstrations of the centers by the teacher cadre were success- 
ful. 

In February 1970, Model Cities agreed to supply the money to allow the project 
to contract with Blaschke again for badly needed management support in both 
operations and planning. A continuation grant proposal was due in April to USOE 
showing the Phase II plan, and a turnkey analysis was necessary to support the 
turnkey planning. A composite of the turnkey plan was laid out in the continuation 
grant and formalized in the evaluation design for 1970-71. 

The overall purpose of the turnkey component was to transfer the techniques 
proven successful in raising reading and math achievement for potential dropouts 
into regular classrooms as a demonstration pilot program. There would be reading 
and math classes spread over the three junior and two senior highs. During 1969-70 
many observers of the RLC wondered whether the students who left the center for 
the regular classrooms would not backslide into the same problems that originally 
led them to be assigned to the RLC. The turnkey classrooms were designed, in part, 
to solve this problem by carrying the RLC procedures into the regular classroom. 

The target population would be all the Phase 1 participants in the Rapid Learn- 
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ing Centers (grades 8-12) who had gained at least one grade level in reading or math. 
It was expected that 174 students would qualify in reading and 129 in math. But 
these classes would also include regular students, both to be a model for their RLC 
peers and to familiarize some regular students with the program in preparation for 
full implejmentation for 1971-72, This decision was a significant modification of the 
turnkey concept, since many more variables were introduced than would have been 
if the classes had contained only students from the RLCs, 

The Evaluation Design of June 30, 1970 contained a summary chart listing 
specific objectives for the turnkey phase of the 1970-71 program. Although this list 
set forth the basic structure of the program as it was to be conducted in 1970-71, a 
number of the statements were worded ambiguously and some of the objectives were 
unrealistic. These were later changed in the Mid-Year Evaluation Report of Febru- 
ary, 1971. The following list sets forth the major specifications as they were first 
stated or later revised, if necessary. 

The key objective for the program (at first ambiguously worded) was: 

The achievement level in mathematics and/or reading of at least 200 poten- 
tial dropouts in grades 8-1 2 :n will be raised 1.0-1. 9 grade levels in 140 days 
of instruction. 

A curious statement was that: 

The cost effectiveness of student achievement in mathematics and reading 
will increase by 50 percent. 

Apparently this was to be determined after the following objective was realized: 



To develop a performance budgeting system that will provide a cost effi- 
ciency ratio. 

Added to this, the efficiency of the instruction was to be checked by determining 
whether 



... 75 percent of the students enrolled showed at least 1.0 grade level in- 
crease in mathematics and in reading in one year. 

31 Originally the spread was 7-12; the writer apparently forgot that no RLC students were in the 6th 
grade the preceding year. 










The specification of procedur es for student selection was also later revised: 

( To identify all the target population eligible to participate in the Turn- 
key program as defined by the following criteria: 

(a) Students in 1969-70 Rapid Learning Center program who gained one 
or more grade levels in reading comprehension and/or mathematics, and 

(b) students outside the target population scheduled in the same classes 
as successful 1969-70 RLC students, 

(2) To select from the target population a minimui A of 150 32 students in 
mathematics and 150 3a students in reading to be enrolled in the Turnkey 
classes. 

It is not clear whether the random selection was to be made from all of the students 
identified in (1) or only from the students identified in (l)(b). However, this was later 
clarified, as stated in the Final Evaluation Report for 1970-71, by simply deleting 
(1Kb). E,s i so, with only 145 students meeting criterion (l)(a), there was a shortage 
of turnkey students. These students were simply scheduled as evenly as convenien t 
among the 15 turnkey teachers, with almost all classes coming from the low-ability 
track. 

Other specifications are easier to interpret: 

• Turnkey programs will be operating in a minimum of ten English and 
eight mathematics classes in grades 8-12, 

• To select fifteen teachers and four aides to operate Turnkey classes in 
grades 8-12, 

» To establish two exit testing dates . . , (January and May) 

• To utilize equipment in Turnkey program with not more than 10 per- 
cent "down” time, 

« To involve effectively a minimum of six groups in the planning and 
operation of the program. 

• To develop and operate a student information system that provides data 
on number of dropouts, school absences, grade retention, and subject fail- 
ures, 

• English and mathematics teachers will be trained to operate Turnkey 
program i* > - ’- y '.ose III. 

32 Revisei -o rb ;■ * in February from 250. 




And, as a result of all this, not only would achievement scores go up, but: 



• The dropout rate in grades 8-12 will be reduced to 5 percent. 

It was unrealistic tc expect the turnkey program to meet its basic goals (improved 
student achievement and lowered dropouts) as well as accomplish such sophisticated 
feats as the development of a student information system. It was particularly un- 
realistic in view of the budget available and the probability that the contracted 
program for the learning centers would again soak up most of the attention and 
resources, as it had done the preceding year. 

Turnkey in Operation 

The following material draws heavily on the Final Evaluation Report for 1970- 
71, notes taken from observations of the turnkey classrooms, and interviews with 
turnkey teachers and administrators. 

The turnkey component was under the direction of Dr. Lewis Lemmond, who 
was formerly with the Liber ty-Ey lau schools. During 1970-71 he was employed by 
the Region VIII center (see Appendix G), which assigned him to Texarkana as part 
of its management assistance effort. The turnkey component received 16 percent of 
the budget for the total program. 

There were 24 turnkey classes: 12 in math and 12 in English, There were also 
15 classroom teachers and 5 aides, one for each of the 5 campuses involved (College 
Hill Junior High, Jefferson Avenue Junior High, Arkansas Senior High, and Liber- 
ty-Eylau Junior and Senior Highs. These classes served a total of 145 former RLC 
students. This was already considerably less than the 300-odd turnkey students 
implied in the continuation grant (if the target students were to be half of the 
turnkey population of 600) and quite a bit less than the 250 implied in the evaluation 
design for a population of 500 (step (2) in the procedures for student selection). 

Students who had been in the RLCs during 1969-70 and who had improved by 
at least onj grade level in either reading or math on the tests given in May 1970 
were "in the turnkey target population,” according to Lemmond. This meant that 
students well below normal grade level could be in turnkey classes. If a student had 
improved sufficiently in only one subject, he was very likely to stay in the RLC for 
the other subject. Also, we cannot be sure that all students who improved at least 
one grade level did, in fact, enter a turnkey classroom, rather than stay in an RLC 





or enter a regular classroom, because of the practice of basing student selection on 
the results of the invalid May test scores. There were no apparent provisions for 
retesting these students. 

Beyond this, there was no special system for assigning students to classes, al- 
though Lemmond did imply that attempts were made to group students with similar 
gain scores. Most of the turnkey teachers interviewed, however, felt that the spread 
of capabilities in their classes was very wide. For example, in one 9th grade English 
class there were students reading all the way from 2nd to 9th grade levels, according 
to the teacher’s assessment. One teacher also surmised that troublemakers were 
funneled into the turnkey class. 

Each of the turnkey teachers was specially selected for the program by the 
building principal. The basis for selection was a set of guidelines provided by Lem- 
mond that suggested desirable characteristics: empathy for students, ability to diag- 
nose individual learning difficulties, and belief in the need for change. The turnkey 
"reading” teachers were former English teachers, as reading had not been previ- 
ously taught at the secondary level. The teachers observed and interviewed were 
quite disparate in terms of their apparent ability to maintain control of the class 
(although this has not proved to be a strong factor in student achievement in other 
programs) and in their acceptance of the Dorsett machines. Almost all, however, had 
achieved at least a degree of individualization; despite the very late arrival of 
materials and wide ranges in learning capabilities, they were strongly committed 
to the benefits of the individualized approach. 

Each of the turnkey schools had an aide shared by all the turnkey teachers, 
generally three. One teacher, however, who had to share her aide with three others, 
felt this was too many. We observed the aides to be very helpful and intelligent in 
dealing with the classes, substantiating the teachers’ beliefs that aides are practi- 
cally a necessity for such a program. Although no special training was provided for 
the aides, some had been involved in the Dorsett program the preceding year and 
vitally assisted the teachers in establishing the program. 

Teachers were given a one-week training session, before school opened, in Au- 
gust 1970. A reading specialist from Liberty-Eylau conducted the reading training 
and a teacher from nearby Magnolia Jr. High, who had worked with the program 
in mathematics, conducted the math training. During the training, the teachers 
became familiar with the materials that would be available by grade levels (tney had 
heard company representatives on June 17 ). For mathematics, these levels wore 
keyed to the diagnostic test. The teachers went through the materials in reading, 



making up the concept hierarchy themselves and keying materials to the concepts 
they taught. They expressed enthusiasm about the training. 

Further training was given in December, There was a two-day seminar on 
behavioral counseling techniques led by Dr. Ray Hosford of the University of Cali- 
fornia, Santa Barbara. The teachers appeared to be pleased, as indicated by a ques- 
tionnaire, because they had been having difficulties in handling behavior problems. 
Additionally, there was a half-day training session on the use of the materials for 
the SRA Diagnostic Reading Laboratories. This session was delayed because the 
materials ordered in September did not arrive until December or January. 

Apart from these sessions, the teachers had to rely on past education and ex- 
perience to develop their teaching techniques. Most of the teachers who commented 
on the training felt that although the initial training was helpful, it was inadequate 
and there was not enough follow-through. It would also have been helpful to have 
training for the aides and to have opportunities to exchange views with teachers in 
the program at other schools. 



Turnkey Classrooms 

The turnkey classrooms, contrary to the original concept of turnkey, were not 
replications of the Rapid Lear .ing Centers nor even of the materials and techniques 
used in them. Instead, certain regular classrooms (with desks in rows) were desig- 
nated as recipients of some of the RLC alumni and of some materials and equipment 
similar to those used in the RLCs. Each, for example, had Dorsett M86 machines and 
programs, but these were used in only a few of the classrooms. For one thing, many 
of the graduates from the RLCs knew the Dorsett programs too well; therefore, new 
programs would have been needed to retain their interest. If Dorsett had stayed, he 
could have produced the needed programs. For another, most of the teachers did not 
know how to keep the machines in running order, and unreliability eventually put 
most of the machines out of commission. And many teachers were soured by the 
initial difficulty of getting tables built and outlets installed to handle the machines. 
None of the turnkey teachers used the extrinsic rewards of the RLCs, and no effort 
was made to have teachers replicate RLC procedures. Thus, Dorsett’s system was not 
turnkeyed. 

In reading, the SRA Diagnostic Reading Labs were used the most, supple- 
mented by Read magazine and the Reader's Digest, the SRA Pilot series (advanced), 
and English for Me (programmed English). In one school all of the reading teachers 
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used EDL materials made available by the Model Cities program in a reading lab. 
Turnkey teachers in math used SRA math labs, computation skills labs and puzzle 
boxes, Sullivan workbooks, and Job Corps materials. They also used regular texts. 

The major difficulty with the materials was that they were so late in arriving 
that teachers could not individualize their teaching until after the first of the year. 
In the midyear evaluation report it was noted that the materials to be used in the 
turnkey classrooms were ordered before school started but did not arrive until just 
before Christmas. This could, in large part, have accounted for the evaluator’s 
further statement that "Instruction is not individualized.” Some kind of self-instruc- 
tional materials are mandatory to achieve individualized teaching in a fairly large 
class with a wide spread in capability. 

The same evaluation report also noted that the excessively late delivery of 
materials was "related” to "administrative problems”: 

There is a lack of unity of purpose and philosophy among administrators of 
various levels within the school districts. This results in a number of prob- 
lems, especially in decisions concerning the budget and expenditures of Title 
VIII monies. 
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Hence, as in the first-year program, lack of a strong focal point for the program 
severely hampered the turnkey effort. 

Some turnkey teachers had set up their own classroom management schemes 
before school started but were unable to use them because the required materials 
were not available. They did, however, administer some diagnostic tests and tailor 
instruction to student capability as much as possible. After the materials arrived, 
it took a few weeks to get the individualized approach working, and by the time of 
our observations in May many of the teachers were operating individualized class- 
rooms. Students worked independently, calling for special help where needed, and 
showed evidence of a responsible attitude toward their own work. But the teacher 
approaches were still diverse. In one class students helped each other; in one, the 
teacher would treat the class as a group if many students were having the same 
problem; and in one a teacher used student leaders to direct the groups. Most of the 
classes, however, did exhibit a much closer relationship between teacher and stu- 
dent than in the regular classes. 

To further complicate the teachers’ task, the selection and scheduling of stu- 
dents for the turnkey classrooms was not completed until nearly two months of the 
school year had elapsed. This came about because no one knew exactly which RLG 




students would be eligible for turnkey until the preceding June, and a great deal of 
rescheduling was required. This implies that the summer was essentially wasted and 
that the rescheduling' did not get under way until school opened in the fall. There 
were also fewer former RLC students than planned; 185 were reported in the Mid- 
Year Evaluation and only 145 in the Final Evaluation, whereas 250 were expected, 

A number of the teachers were intensely involved with the program. One con- 
tinually pressured the administrators to obtain the missing materials, another 
sought specially programmed materials for her class. Frequently, teachers noted 
startling improvements in the attitude and behavior of individual students,. Some 
teachers expected equally startling gains in student achievement (as much as three 
years) and were completely sold on the program. 

The students, on the other hand, seemed to be less pleased. In May, the teachers 
said that many students thought of turnkey as the "dumb class” and some requested 
permission to leave. Teachers reported that students demanded traditional grades, 
rather than progress indicators— another evidence of their desire not to be specially 
singled out by the program. If troublemakers had, in fact, been put into the turnkey 
classes, it would not be surprising for students to feel stigmatized. Despite this, some 
of the students evidenced genuine interest in the materials and (in a few instances) 
the machines they had been working with. 

Turnkey Achievement 

The overall system-wide objective of a dropout rate of 5 percent or less in grades 
7-12 was achieved; the actual rate was 4.3 percent compared with 8.3 percent the 
year before. No attempt was made to assess how much turnkey contributed to this 
common goal as compared with the Rapid Learning Center component. The favora- 
ble impact on the dropout rate again stands out, as in the 1969-70 school year, as 
the principal positive result, since the 1970-71 achievement scores were anything 
but encouraging. 

The only specific measure of turnkey academic performance was the difference 
between the pre- and post-test scores on the ITBS (forms 5 and 6), but this test was 
given only to learning center students and thus to only those turnkey students that 
were also in the learning centers. For reading, this was about 45 percent of 111 
turnkey students, and for math about 28 percent of 79 turnkey students. Because 
these are students who did worse than the other students in the turnkey program, 



since they were kept back in the RLC for one subject, this sample is probably biased 
toward too conservative an estimate of true gain from the turnkey classrooms. 

Only 24 percent of turnkey students tested achieved or exceeded the objective 
of 1.0 or more years gain in Reading Comprehension, and only 55 percent achieved 
or exceeded 1.0 years gain as measured by the Total Arithmetic score. Also, 43 
percent of the reading gains and 25 percent of the math gains were negative, suggest- 
ing that over a third of the turnkey students tested were doing a significant amount 
of guessing. But it is comparison with other groups of students that truly makes the 
turnkey academic results look bleak. 

The control or regular group was limited to those students who were taking one 
RLC subject but not both, and were not in the turnkey program— or about 80 
students in regular classrooms for reading and 80 for math. This limited design was 
presumably dictated by testing convenience. For each of the five subtests of the 
1TBS, the Final Evaluation Report presents tables of the three two-way comparisons 
between RLC/RLC students, RLC/turnkey students, and RLC/ regular students. 133 
For each comparison, the two groups were chosen to equate pre-test scores and IQ 
scores. This selection yields an N of only 18 for the turnkey students out of the 
population of 145, raising serious doubts about whether results from so small a 
sample can be generalized. 

There were no significant differences (by t test) among the groups on the 
Vocabulary and Reading Comprehension subtests. Thus, 18 turnkey students did no 
better than 82 regular students, but 239 learning center students also did no better 
than the regular students. For arithmetic, turnkey students did significantly better 
(0.02 level) than learning center students on the Total Arithmetic subtest (an aver- 
age of the other two arithmetic subtests), and noticeably but not significantly better 
than the regular students. 34 Thus, for this small sample, there is no demonstrated 
advantage of turnkey procedures over regular classroom procedures. However, with 
the large number of negative gains noted above for these gains on standardized test 
scores, the reliability of the results is doubtful. Validity is also a major question, but 
there was no attempt to provide any criterion-testing of the turnkey students. Thus, 
while decisionmakers have reasons for looking skeptically at turnkey classrooms, 
there is no clear basis for deciding what changes are needed. 

35 Roberts and Andrew, 77/ • Texarkana Dropout Prevention Program Final Evaluation Report, Region 
VIII Education Service Center, Magnolia, Arkansas, August 1971, pp. 73-80a, 

3 < Ibid., p. 80b. 
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Turnkey Conclusions 

The turnkey classrooms were different from the conventional Texarkana class, 
particularly after the special materials arrived in January. Four differences stood 
out: (1) more diagnostic testing, (2) more attempts to prescribe individualized 
material, (3) more material and more individualized materials, and (4) a more infor- 
mal relationship between teachers and students. In many classes individualization 
has gone only part way, for reasons previously discussed. In short, while turnkey 
classrooms were different from the conventional Texarkana classrooms, they were 
also different f rom either of the learning systems used in the RLCs. Since no attempt 
was made to turnkey Dorsett’s materials and procedures as a complete system, and 
since Dorset! had no role m implementing the turnkey phase, he could not make the 
program modifications and provide the extensive teacher training that would have 
been required to integrate his system into the regular school program. There were 
also doubts about the effectiveness of the Dorset t materials, but no solid evidence 
pro or con. It was accordingly decided to use Dorset! machines as only part of the 
program, and to have other materials provided by the component manager organ- 
ized into a curriculum structured according to behavioral objectives worked up by 
the teachers themselves. 

Thus, the turnkey effort now looks like one of the many compensatory educa- 
tion projects of the last few years. However, unlike the most successful compensa- 
tory projects, there was little intensive effort to give continuous in-service training. 
With no forum for common problems, teachers did the best they could, and the 
results depended on the quality of the > eacher (or sometimes the aide) alone. The 
unpredictable but severe lack of materials for the entire first semester made in- 
dividualizing instruction very difficult. Add to this problems of over-large classes at 
the beginning, and it is amazing that some of these classes finally did work well. 

Not foreseeing the importance of regular in-service sessions was probably due 
to lack of experience of the component manager with similar projects and to the 
difficult logistics of working with five schools (four widely separated). Additionally, 
the attention required for resolving the Dorsett contract and selecting a completely 
new contractor took much management time from the turnkey effort and sharpened 
the difficulties arising from diffused responsibility and lack of a strong focal point 
for decisionmaking for the program as a whole. 

Dropouts declined, students seemed to learn, there were often fewer discipline 
problems at the school, many teachers liked the individualized approach, and the 
children responded to individual help. However, the small sample of students tested 



did not meet the objective of 1,0 or more years gain, and they did not score signifi- 
cantly better than an equated group of regular studen ts. There are serious problems 
of statistical reliability and validity, but the achievement results are not encourag- 
ing. 

The turnkey program also failed to meet several other objectives. There was 
insufficient testing for a true evaluation of turnkey achievement, and no cost data 
were collected at all. Also, due to time and scheduling problems, no one but the 
turnkey teachers themselves and the school counselors received training in in- 
dividualized instruction and behavior management. This places quite a burden on 
the training function in trying to expand turnkey in 1971-72, 

In sum the turnkey classrooms did have a different curriculum from the conven- 
tional Texarkana classroom. Performance contracting did turn out to be a change 
agent in Texarkana, The educational process implanted, however, is far from the 
advanced technology, advanced classroom-management techniques, and nongraded 
individualized systems that many people hoped performance contracting programs 
would introduce into standard classrooms. 



RAPID LEARNING CENTER COMPONENT 



The RLC Plan 

The general objectives were to reduce the percentage of dropouts significantly 
and to provide more efficient reading and math instruction in grades 7-12, The 
specific project objectives were: \ 

• The math and/ or reading achievement levels of 300 potential dropouts 
would be raised by 1.0 to 1.9 grade levels in 140 days of instruction; 

• The dropout rate in grades 7-12 would be reduced to 5 percent; 

• The cost-effectiveness of the system would increase by 75 percent In stu- 
dent reading and/or mathematics achievement; 

• 22 teachers (English and math) would be trained to operate the RLCs; 

• A public information system would be established serving essentially all 
interested or affected groups. 



There were also some 16 process objectives serving as guidelines to meeting the 
goals above. As with the turnkey objectives, the achievement and cost-effectiveness 
objectives were ambiguously worded initially, and the overall goals were perhaps 
overly ambitious. 



RFP end Contract 

Educational Developmental Laboratories, Inc., a division of McGraw-Hill, was 
chosen on the basis of a competitive bid to he the LSG for 1970-71. The contractor 
selection form attempted a quantitative assessment of each potential contractor and 
his system, whereas the selection form in 1969-70 had essentially been a checklist 
of points from the RFP. The RFP that Texarkana submitted to prospective LSGs in 
the summer of 1970 differed from its previous RFP in two important respects. First, 
the evaluation design and data reporting requirements were outlined. Second, the 
RFP stated that performance would be measured by the results of standardized tests 
and also by the extent to which program objectives were realized. The proposed 
payment arrangement flowed from this approach. The RFP stated that 75 percent 
of the payment would be based on standardized norm-referenced tests and 25 per- 
cent on the achievement of interim and final objectives. The RFP also stated that 
the contractor must be prepared to guarantee that the dropout rate would not 
exceed 5 percent. 

The contract payment provisions were derived from a procedure EDL proposed, 
based on estimated project costs of $65,788. This was divided into two parts. One part 
was called the Fixed Charge and set equal to $89,012. This sum was to be paid to 
EDL in two equal installments, at the time the contract was signed and in December 
1970, 

The remaining part, $26,776, was to be held in escrow until the student achieve- 
ment results were available and the contract was settled. All or part of this sum 
would go to EDL depending upon a penalty-bonus calculation based on a point 
system, (The formula is in the contract in Appendix F.) This formula multiplies by 
4 the number of students that qualify for the achievement guarantee in order to 
compute the total number of points to be allocated. This total was to be divided into 
$26,776 to get the dollar value of a point. A point calculation was to be made for each 
student. No points were to be assigned for satisfactory completion (scores between 
75 and 85 percent) of the criterion-referenced tests and for reading and mathematics 
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achievement growth in the range of 1.0 to 1.9 years. If the student norm-referenced 
achievement gain were less than 0.9 in mathematics or reading, 3 penalty points for 
each subject would be assigned; likewise, if growth were greater than 2.0 years, 3 
bonus points would be assigned for each subject. Scores on the cr ; arion referenced 
tests of less than 75 percent would earn 1 penalty point per subject, and scores 
greater than 85 percent would earn 1 bonus point per subject. 

To compute EDL’s payment, one would begin with the $26,776 and subtract the 
number of penalty points multiplied by the value of a point. He would then add the 
bonus points multiplied by the value of a bonus point. If more bonus points than 
penalty points were earned, EDL would receive $26,776 plus a token of $1,00 for each 
student with exceptional performance. 

EDL’s response to the Texarkana RFP proposed not to constrain the total bonus 
it could receive, The final contract, however, included the constraint, which gave 
Texarkana more assurance about the maximum payment it might have to make. 
Also, EDL had proposed earlier to accept penalty points for all dropouts, including 
dropouts within the target rate of 5 percent. The contract, however, merely stated 
that EDL would be "responsible” for dropouts after the first two weeks and payment 
was not linked to the dropout rate. What this responsibility entailed has not been 
spelled out. 



Rapid Learning Centers 3S 

EDL equipped five centers for grades 7-12 under the dropout prevention pro- 
gram and one center for the 6th grade at Washington Elementary under the Model 
Cities program. This last center will be discussed later. There were 426 students in 
the 5 dropout prevention centers and 110 at Washington School’s sixth grade. The 
trailers and classrooms that had been modified to Dorsett’s plans were used with 
only slight environmental changes The instructional program was built around the 
EDL Learning 100 system of hardware-intensive instruction, using very detailed 
materials. The system used individualized or small-group instruction without ex- 
trinsic motivators or unusual student incentives. 

Each center had a teacher and an aide (except for Washington, which had two 
teachers and an aide), and the original teachers and aides were given 40 hours of 
special instruction. Although teachers and aides were on the EDL payroll, their 

08 See also Edmund Uazzera, "A Contractor’s Viewpoint,” Compact, Vol. 5, No. 1, February 1971, pp, 
18-16, for more detail on the centers. 
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salaries were determined by the Texarkana School District salary schedule and they 
were subject to the policies and administration of the LEA. EBL was represented 
by its Component Manager, Mr. Edward G. Miller, a former Texarkana junior high 
school principal. 

The target population consisted of students with IQs greater than 75 who were 
two or more grade levels behind in reading and/or mathematics, Students could be 
scheduled into the RLCs for an hour a day per subject in either or both subjects. 
Twenty students at a time were scheduled into the centers. One of the project 
personnel estimated that between 50 and 60 percent had been in the centers last 
year. 



RLCs in Operation 

In general, the centers have operated as smoothly as most special programs, and 
particularly well considering the handicaps attendant on discontinuing one contrac- 
tor’s involvement and bringing another in on relatively short notice. The start of the 
1970-71 program was delayed by at least two months by this change of contractors. 

There was teacher turnover. The EDL system is quite challenging— as were 
many of the students in the centers. Five people out of the or iginal 13 were replaced 
(four teachers and one aide) and five new people trained on the job. Interviews at 
the centers indicated that this training was co t >rod very effective, and the re- 
placements all seemed above average in ability , i'he discipline problems at one of 
the high schools ended very quickly, and all the students improved their work. 
Miller commented that the group had stabilized most satisfactorily, and he felt this 
group was well prepared for next year’s program. 

Student selection and scheduling was a problem. The lack of good, easily ac- 
cessed records of IQs and reading test scores made it difficult to determine which 
students should be assigned to the centers. The coordination of center attendance 
with other class schedules was also a problem. At the high school level, students 
have some flexibility and choice about schedules so the scheduling difficulty was 
exacerbated. The result was that, as the evaluator put it, for several weeks there was 
a "fluid” population in the centers. Student selection and scheduling seems to be an 
endemic problem with most performance contracting programs. If great attention 
is not paid to it, a significant number of scarce teaching days can be lost. 

Testing was also a problem. The evaluator noted that the conditions under 
which the standardized norm-referenced tests were administered were far from 



ideal. There was a lack of proctors, students did not have enough test-orientation, 
and the switch in the contractor and a decision to use a different form of the ITBS 
delayed test administration. And as the evaluator noted, the testing required a great 
deal of management time. Again, such problems seem to be common throughout 
performance contracting. 

The impressions we formed on seeing the centers in operation were almost all 
positive. In the five (of six) centers visited at least briefly, the teachers were all well 
organized and had pleasant or warm relationships with the students. This was also 
true of most aides. Quite a few aides taught the reading and math classes alternately 
with the teacher during the day, an indication of their competence. Attendance in 
the classes observed was always well below 20 (on one day, one class had only 6), 
although some teachers mentioned having a class or two that were nearly full. The 
machines worked most of the time. They were tended by a maintenance man, and 
the variety of activities in the EDL program made it easy to program around ma- 
chine failures. EDL strongly urges that the teachers be approachable by the stu- 
dents, and this practice seemed quite successful. Several teachers commented on 
changes in work rate and behavior they noticed after they had continually en- 
couraged students and listened to their problems. Another indicator of this concern 
for students is the approach to the machines; one teacher told her students at the 
beginning, "We have these machines because no one person knows everything, and 
they’ll help us help you.” Naturally, some students work rather slowly, but all could 
collect and set up their materials for the next step, and they worked throughout the 
period. The evaluator reported that "some” students felt the centers were "just for 
dummies,” but we did not hear about such reactions in our teacher interviews. In 
sum, one could guess that this was in fact a system that had been used in many other 
places and shaken down. There were curriculum weaknesses, especially in math, but 
that was also a much newer area for EDL. 



RLC Achievement 

As noted in the section on the turnkey phase of the program, the primary 
objective of lowering the dropout rats to 5 percent or less for grades 7-12 was 
achieved. Since the learning center component involved 426 of the total of 505 
students, it probably deserves most of the credit for this achievement. 

By contrast, the performance on standardized tests was dismal, raising doubts 
about the assumed relationship between improved academic achievement, as mea 



sured by norm-i "'renced tests, and dropout prevention. The ITBS Forms 5 and 6, 
resnectiveiy, were administered as pre- and post-tests to 251 students for reading 
and about 261 for math (some students, of course, took both, so the total is greater 
than 426). The evaluator noted that few students were eliminated from the program 
for reasons other than dropout, so the sample teste' 1 is essentially the entire learning 
center population. 

Only 29 percent of the RLC students achieved or exceeded the objective of 1.0 
or mo re years gain in Reading Comprehension, and only 25 percent achieved or 
exceeded 1.0 years gain in Total Arithmetic (the two subtests on which contract 
payment is based, although the other subtest results are similar). The mean gains 
(computed after the fact for the cost analysis) were 0.48 years in reading and 0.81 
years in math including the sixth grade as well as secondary school students (7-12). 
This is a far cry from the major gains forecast by the contractor, and raises doubts 
about whether EDL recovered all its costs. Also, 41 percent of the Reading Compre- 
hension gains and 33 pe cent of the Total Arithmetic gains were negative, raising 
serious questions about the amount of guessing and therefore the reliability of the 
testing. This seems to be an endemic problem in using standardized tests for low- 
achieving populations. 

To make matters worse, the RLC students showed no significant differences in 
gains over either the 81 "regular” students or the 18 turnkey students. As the 
evaluator summed the results, "The desired achievement gain in reading and math- 
ematics was not accomplished by the performance contractor, EDL/McGraw- 
Hill.” 36 

There was a criterion-referenced testing program as well in 1970-71, allowing 
a comparison of results on criterion and norm-referenced tests. However, the criter- 
ion-referenced testing program encountered severe difficulties. The contract called 
for EDL to submit, within 00 days of the signing of the contract, a detailed list of 
performance objectives and a pool of criterion-referenced test items. For each per- 
formance objective, at least five questions had to be submitted. Meetmg the time 
requirement turned out to be impossible; EDL did not finish the task until January. 

The original idea had been to administer four tests and utilize pre- and post-test 
scores, but the difficulty in obtaining test questions ruled this out. However, since 
the EDL contract based the 25-percent criterion payment only on the final criterion 
tests, this was not a serious problem for contract settlement purposes, but it did 
ignore the important question of gains in performance. 
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38 Op. cit, p. 116. 



Two test dates were scheduled, January and May, with payment to EDL based 
only on the May results. After EDL submitted the five questions per objective to the 
evaluator, he constructed the January test by sampling items for almost all objec- 
tives. The only objectives ignored were those involving teaching machine work, for 
winch it was not feasible to test students. The wording of the objectives required that 
all instructions be given orally for objectives below a particular level in the cur- 
riculum. It was decided to use the laboratory directors as test administrators, and 
to test all students (grouped by level) in the same half-day. Some of the teachers have 
given vivid accounts of this experience, W.th no chance for field trial, the test was 
far too long; reading often took the full morning (2-1/2 hours) and math another 
hour. Not only were there too many items, but having to give oral instructions to 
the group required waiting for the slowest child on each item. The students were 
very tired by the end. Some instructions were unclear, and teachers had no oopor- 
tumty to resolve ambiguities because they had not seen the test before giving it. 

Because of these difficulties, it was decided to treat the January test as a trial 
run, and to construct a much shorter test for the beginning of May. This time, with 
many fewer items available from the contractor, the evaluator decided merely to 
sample the objectives covered, with a few items per objective. With a shorter test, 
it was possible to administer it more than once at a given level, and thus use project 
personnel as test administrators. In May, there was a preview session for the ad- 
ministrators, and the testing went off with barely a hitch. 

The results were in rsharp contrast to the scores on the norm-referenced test. 
About 78 percent of 336 students (including Washington sixth-grade students) tes ted 
in reading and 66 percent of the students tested in mathematics correctly answered 
75 percent or more of the objectives tested. However, the evaluator pointed out that 
the test was too easy, because it sampled only a few objectives and took only about 
an hour to administer. Further, with only a sampling of objectives, he noted that 
reliability was worse than for a standardized norm-referenced test, and the content 
validity was highly questionable. He recommended that criterion-referenced testing 
should be made more comprehensive and should be more integrated with the. in- 
structional program. 87 

Again, Texarkana experience is in accord with the experience in other dis- 
tricts. 38 Developing criterion-referenced tests is back-breaking work, and there are 

" Op. cit., p. 57. 

s ® See particularly Polly Carpenter; Case Studies in Educational Performance Contracting: Norfolk, 
Virginia, The Rand Corporation, R-900/2-HEW, December 1971. 



as yet no data on the reliability of test items. Attempting to test all students and 
all objectives is so monumental a task that some sampling procedure, perhaps as a 
part of the normal instruction, seems the only feasible approach. This makes 
achievement measurement depend on careful statistical computations. In short, the 
criterion-referenced testing procedures are only now being developed; the state-of- 
the-art leaves much to be desired. Nevertheless, such tests can be more valid meas- 
ures of the short-run effects of instruction. Their relation to longer-term goals has 
not been explored. 

RLC Program Cost 

The Addendum to the 1970-71 Final Evaluation Report provides a rough esti- 
mate of the cost-effectiveness of the learning center program compared with regular 
classrooms. The unit of comparison was cost per grade-level increase. Before report- 
ing these comparisons for reading and mathematics, we should note several deficien- 
cies. First, there is no mention of the test used to comp ute gain scores for the regular 
students. One can only assume that the standard district tests (SRA and ITBS) were 
used, raising a serious question of the comparability of the gain scores between 
regular and learning center students. Next, no supporting resource and cost data are 
presented to indicate how the total program costs were computed. The resources 
shown in Table 4 represent our guess as to the components of the learning center 
program, but do not include supervision cost for either LEA or LSC, and do not 
include other elements relevant to EDL’s program cost. The reasons for including 
or excluding a given resource are crucial to determining the adequacy of a cost 
analysis, and their lack is a serious omission. Finally, there is no indication of what 
kind of cost is being computed, i.e,, how much is fixed and how much is variable with 
the number of students, and how much is acquisition jost versus operating cost. 

Nonetheless, we report the Addendum costs since they are the only ones availa- 
ble and were in fact used for decisionmaking by Texarkana. It can be seen from 
Table 5 that the learning center program uniformly cost more per grade-level in- 
crease than did the regular program. But note also that the learning center popula- 
tion consists entirely of low-achievers, whilw the regular population has the full 
range of ability. Thus, it is not clear that a comp orison of costs for the regular and 
compensatory programs is relevant. It would be ideal if we knew the average cost 
per grade-level increase for regular students of comparable ability to the learning 
center students. However, the cost differences are so large except for secondary 




Table 4 

PROGRAM AND RESOURCE INFORMATION FOR 1970-71 



Charaa ter ‘ s tics of siud&n 


ts Grades 7-12; educationally handicapped 

(at least 2 years below grade level) IQ 
at least 75 


Program scope 

Number of students . , . . 
Class time. 

Class siEC, 




Facilities 

Space , * 

Utilization, 

Furnishings, 


1000 sq ft 


Staffing 

Certified teachers 

Special teachers 
Faraprofessionals ...... 

Other staff, 




Equipment 


Flash-X 


Materials , , 


strips 


Pre-service training 




In-service training 




Other support 




Incentives 





a Model Cities funded 110 Sth-graders for the same instructional program, 
giving a total program of 395 students. 
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SOURCE: 'Taken, from the Addendum to the Final Evaluation Report, August 



reading that it is quite likely that much a comparison would still show that the 
learning center v as more expensive per grade-level increase. The secondary reading 
cost for the learning center looks like a very favorable investment for an undera- 
chieving population— that is, until we look closely at the average grade-level in- 
creases given in Table 5, It is quite unlikely that the 0.27 listed for the entire 
secondary English student population is accurate, and the 0.35 for mathematics is 
not much more convincing. If they are accurate, then the reliability and especially 
the validity of those standardized tests are very questionable.. 

In sum there is again little information that a decisionmaker can use with 
confidence. However, the costs of the Texarkana program expressed on a grade-lev 
increase basis is not particularly attractive. 

RLC Conclusions 

The dropout rate was lowered from 8.3 percent to 4.3 percent, well surpassing 
the primary program goal of a reduct 4 m to 5 percent. However, with more than hall 
the students gaining less than 0.5 grade levels in all areas, and a large number of 
students apparently guessing on the norm-referenced tests, it would be rash to 
attribute the difference in the dropout rate to improved academic performance. 

Academic improvement of 1.0 grade levels o: better was also a goal, so the 
failures of two years of performance contracting to achieve these results could not 
be ignored. Texarkana therefore accepted the evaluator’s recommendation that the 
LEA operate the learning centers instead of contracting with business firms. It is 
hoped that the center operation will have more teacher and administrator input and 
support in the future program years, and that these changes will lead to better 
results. The first attempt to provide a cost analysis— drawing on contract support 
—did not furnish the best basis for desisionmaking. Testing remains another prob- 
lem area. Even with one year’s experience, the testing program still was not ade- 
quate in the second year. 

Attempting criterion-testing took valuable effort away from improving the reli- 
ability and validity of noim-referenced testing, even though the normed tests have 
inherent defects. 

Finally, far too much was attempted for the numbers and experience of the 
available staff. Most attention went to the performance contract program to the 
exclusion of turnkey and other components. Perhaps this was the major reason that 



learning center achievement received most of the eva luation attention, while turn- 
key achievement, of potentially greater significance to the district, received less 
Yet the end products of attempting such ambitious goals are likely to be far 
more revolutionary than they would have been under a cautious choice to cling to 
evolutionary growth under Title I and small-scale innovations. In some form, in- 
dividualized learning management systems, program budgeting, criterion testing, 
and a student information system have all taken root. And curriculum revision, 
integrated vocational education, and individualized counseling are still included in 
the goals. 



MODEL CITIES SUPPORT 

The 1970-71 Model Cities programs were largely designed at the time of the 
Continuation Grant Proposal in the late spring of 1970. There had been a desire from 
the first year to try out performance contracting at the elementary level. This desire 
stemmed from a belief that dropout prevention would be more effective if remedial 
academic work started earlier. Supporting this expanded effort was a redesigned 
continuation of the Reading Clinic Program; and of interest in its own right was a 
turnkey version of the adult GED program. 

The first program was an extension of the main performance contract to an 
estimated 250 sixth-graders (who had the same qualifications: two grade levels 
behind and IQ of at least 75), This program was specified in an Addendum to the 
RFP. The only difference from the main contract was a different cost-reporting 
format for Model Cities. Otherwise, EDL (as it turned out) was to use its Learning 
100 program and its Sullivan math in the two classrooms refurbished at Washington 
Junior High last year (but now changed to Washington Elementary, an all-sixth- 
grade school). 

As explained above, there had been two changes of teachers, but by May they 
had settled in very well. They set up one classroom as the reading lab and used the 
other for math, with the aide assigned to the reading laH. The class sizes observed 
were typical of that center, with 12 in math and 13 in reading. In reading, fl of the 
students were only at readiness level and required constant help. Two students at 
the fourth reading level who were making rapid progress also needed attention for 
supplementary work. In math, the students ranged from books 1 to 7 (addition to 



fractions), with book 5 encountered most frequently. Candy and chewing gum proved 
to be effective motivators for slow or difficult students. The reading teacher was 
pleased with the clear progress the new materials seemed to promote as compared 
with her recent student teaching experience. She noted that most other Washington 
teachers are in favor of the program, since they know they cannot help the slow 
students within regular classes. Science teachers and others say they have noticed 
definite progress. 

The second program was a continuation of the first-year Reading Clinic, but not 
subcontracted by the school this year. A brief visit in May 1971 elicited the following 
information. The clinic operated with one white teacher (formerly a home economics 
instructor) and one black aide in a science lab room in the Washington sixth-grade 
school. The program was designed for 69 students in five classes of 12 each, but the 
class observed that day had only 5 students. 

The materials used most were the EVCO Touch and Talk series from the Mod 
ern Methods of Instruction company. The class also used some of the BRL Sullivan 
materials. The materials initially provided were too elementary according to the 
teacher, but nothing supplementary was available until near the end of the year, 
when the Sullivan materials and some others were made available. In the meantime 
she had borrowed materials from other teachers and made do with what she had. 

She stated that the children quickly spaced out in the materials so that it was 
easy to provide individualized lessons with the available materials. The group also 
quickly divided into sets of interested and uninterested students. She had made use 
of some of the teaching machines. At first the children found it exciting to work with 
them, but shortly took them routinely. She felt there was a good spirit in the class 
but there were some problems with some chronic absentees. 

Of the initial group of 50 pupils, 6 of the faster students had been returned to 
the regular classrooms. She surmised that a few of her poorest students had been 
recommended for the program by teachers who wished to be rid of them. She 
administered a number of diagnostic tests. For example, at the start, her students 
scored on the Botel tests from 0,1 of the first-grade level to the 2.3 grade level. Those 
few students at or above the third-grade level were released (presumably into the 
RLG). Her classes were due for post-testing the following week and she was confident 
of progress, but did not mention the criterion she would be evaluated on. 

The third program was a novel extension of the Dorsett GED effort that ended 
in December 1970. Significantly, the "Second Chance” GED program was then taken 
over in January 1971 by the Manpower Development Corporation in something very 
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close to a turnkey fashion. MDC hired new staff and strengthened the curriculum 
by adding some new materials. In response to an evaluation report prepared by 
Blaschke, MDC reduced entrance-testing, gave cash incentives to students mr 
hourly attendance and for passing the GED, concentrated on residents old enough 
to take the GED, and developed an operations manual for the centers. This new 
program was operated from January 1 to July 30, 1971, and initially enrolled 89 
neighborhood residents. After the pre-test, 40 were assigned to the reading and math 
remedial work, and 48 went into the GED preparatory curriculum. 

The program was very successful, with 34 of the 48 students earning GEDs by 
the end of July. Arkansas requires a minimum of 120 hours instruction before the 
GED exam may be taken, but the program students needed an average of only 56 
hours, with a median of only 25. Results for the 18 students in reading and math 
that could be persuaded to take both pre- and post-tests were also impressive: the 
average gains were 1.8 years for reading and 1.4 years for math. This success was 
obtained in an overall average of 67 hours of instruction, using a system of com- 
pletely individual scheduling of time spent at the centers. Despite this success, the 
school board did not vote any funds to transfer the program to school sponsorship, 
so Model Cities was forced to keep it as an experimental program. 

Beyond these major efforts, Model Cities also filled in some important Title VIII 
program deficiencies. One example was funding a training session at College Hill 
Junior High for the turnkey teachers to help them master their newly receive 
individualized materials. This was paralleled by a Title VIII funded turnkey train- 
ing effort at Liberty-Eylau Junior High. , , 

The most recent Model Cities assistance consisted of providing money to the 
Title VIII project to contract out the development of a modest PPBES for use in 
1970-71 and thereafter (the contract is in Appendix I). Model Cities was also heavily 
involved in the program design for the third year (this program will be described 

6 In conclusion, the availability of Model Cities funds has permitted Texarkana 
to avert some serious crises. It also permitted Texarkana to do some program expan- 
sion that allowed them to coordinate their dropout prevention efforts with school 

and community needs. 
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IV. FUTURE PLANS 



A COMPREHENSIVE APPROACH 

After two years of hard work and massive experimentation, Texarkana is trying 
once again to implement the kind of broad approach to dropout prevention envi- 
sioned in the original preliminary proposal. The objectives and components outlined 
in Table 6 are a good illustration of this scope. The programs directed at dropout 
prevention are to be a mixture of academic remediation, academic curriculum re- 
form, instructional method reform, vocational orientation and training, and per- 
sonal counseling. These efforts are to be supported by programs that remedy student 
needs interfering with performance (health, welfare, psychiatric), that provide a 
basic student data information system, and that organize and process evaluation 
data by programs so that they may be evaluated against each other and against 
regular school programs (the PPBES system). 

The vocational courses, the training of school personnel in PPBES, provision of 
peer tutoring and sponsorship of intramural athletics, and specific assistance to 
students with nonacademic needs are all new process objectives compared with last 
year, but they have all clearly evolved from previous efforts. The Texas schools have 
had state-supported vocational programs for some time, and Model Cities funded an 
evolving vocational program for two and a half years to help Texarkana, Arkansas, 
develop capability for a state-funded vocational program that started in 1970-71. 
The superintendents in both districts, with the stimuli of the Title VIII and Model 
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Table 6 

OBJECTIVES OF THE TEXAiUCANA PHASE III PROGRAM 



PURPOSE: 

The Texarkana Dropout Prevention Program, Phase III , is designed 
to accomplish a fourfold purpose: 

1. To reduce the dropout rate in the Texarkana and Liberty-Eylau 

School Districts , 

2* to assist the potential dropout to be successful in school, 

3, to help the potential dropout to like school, and 

4, to demonstrate a model dropout prevention program that mignt 

be implemented in any school * 

METHODS AND TECHNIQUES: 

To accomplish the above objectives, three components will plan, 
initiate, administer*, and evaluate the following activities: 

Curriculum and Instruction Component: 

1- A vocational orientation course will be offered for all 
9th grade potential dropouts. 

2. A vocational exploratory course will be offered for all 
10th grade potential dropouts* 

3. Four hundred potential dropouts will be provided with 
individualized instruction in reading and/or mathematics 
through a performance contract with a private industrial 
firm, 

4* A thorough curriculum study will be made of the offerings 
in mathematics and English in Grades 1*12* 

Training and Supervision Component : 

1, English and mathematics teachers will receive training in 
the techniques of individualizing instruction, 

2, English and mathmatics teachers will receive training in 
behavioral management techniques, 

3, School personnel will be trained in program planning, 
budgeting, and evaluation, 

4* Counselors will be trained in behavioral counseling 
techniques * 

Pupil Personnel Component, 

1, Provide group activities such as peer tutoring and 
intramurals for potential dropout students. 

2, Develop a comprehensive information system. 

3, Assist students with such special needs as health, 
welfare, psychiatric, etc, 

4, Help students in the adjustment process by providing 
comprehensive guidance and counseling services for the 
potential dropout and his parents, 

SOURCE* Dropout Prevention Progrmn (Title VIII y Section 80? 3 ESEA) 
Formal Proposal 3 Texarkana Independent School District, No, 7, Texarkana, 
Ark,, April 30, 1971, 
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Cities programs, have made strong efforts to coordinate their vocational and aca- 
demic programs. With the expansion of the dropout prevention program this year, 
the City is emerging with a comprehensive and coordinated program. The PPBES 
system was designed last year, and the stress is on training the users this year. On 
special needs, the health personnel operate from the same building as the Title VIII 
program, and there has been informal coordination, with Model Cities helping in 
such areas as welfare and mental health. 

Many of last year’s objectives did not get much attention from the overloaded 
staff, so that teacher training and support for counseling to students and their 
parents will be essentially new. The natural focus of interest, therefore, is on how 
things will be handled differently in 1971-72. 



THE ART OF THE PRACTICAL 

The overall product objectives and their performance criteria are: 

1. The students who are potential dropouts in the Texarkana and Liberty- 
Eylau secondary schools will respond positively to the dropout prevention 
program as indicated by a reduction in the percentage of those who drop 
out of school. The minimum acceptable criteria of percent of dropouts 
during the 1971-1972 year will not be more than 5.0 percent for the 1969- 
1970 school year. 

2. During the 1971-1972 school year, the students participating in the Title 
VIII project will have successful achievement experiences as indicated by, 
(a) 75 percent of the group earning one or more grade level increase in 
reading and/ or mathematics on a nationally accepted standardized test; (b) 
75 percent of the group having at least a "C” average in all subjects; and 
(c) 75 percent of the group achieving at least 75 percent of the objectives 
measured by criterion-referenced tests, 

3. During the 1971-1972 school year, the participating students will increase 
their positive attitude toward school as indicated by (a) their monthly 
average daily attendance rate equaling or exceeding the monthly average 
daily attendance rate of all the students reported in the official register for 
that building; (b) a mean gain of 30 points on the School Sentiment Index; 
and (c) at least a 75 percent increase in the number that aspire to complete 



high school as recorded on the Aspirations of Grade Level Completion Test, 
4. The Title VIII staff, in coordination with the school administrators in the 
participating school systems, will demonstrate an effective model dropout 
prevention program as indicated by: (a) achieving objectives 1, 2, and 8, and 
(b) the cost benefits for the additional students retained in school, in terms 
of expected lifetime earnings, will be greater than the operational costs of 
the* Title VIII program. 30 

The main objective is hard to interpret, hut was later clarified to be a dropout rate 
that is 20 percent less than that of 1970-71. The second objective is quite demanding 
for the target population on the standardized tests and the grade average, but 
conservative on the criterion tests. The attendance objective is highly relevant, and 
the aspiration level seems at least useful. The cost-benefit measure is important to 
include, but this is not very realistically defined. 40 Indicative of project cost is the 
expansion of the proposed budget to about $417,000 as part of the planned eff ort to 
spread successes more widely in the schools this year (last year’s projection was for 
$550,000 this year and about $1.9 million overall). 

This increase reflects a combination of maintaining old areas but narrowing 
their focus and of trying the new objectives noted above. Looking at the program 
summary chart (Table 7), one sees that College Hill Elementary has substituted for 
Jefferson Avenue Junior High of last year, reflecting a greater effort at the elemen- 
tary level this year to avoid later academic deficiencies— a very desirable goal in its 
own right. The turnkey, now aptly renamed simply ’'individualized instruction, is 
concentrated in the Arkansas district alone and in only two schools (down from the 
five, in both districts, of last year). Combined with the emphasis on continued train- 
ing of math and English teachers, this intensified effort seems to have a much better 
chance of paying off. The proposal is silent on the question of what individualized 
materials and methods will be used, but does say that the EDL techniques will be 
used everywhere they are "appropriate.” The rumored "turnkey purchase of EDL 



39 Dropout Prevention Program, pp, 95-97, (See footnote to Table 6.) 

40 The cost-benefit measure looks sophisticated at first glance, but is misleading. In the first place, we 
doubt that anyone really knows what the return to schooling in Texarkana is in terms of lifetime income. 
In the second place, since at most 17 percent of the students would have dropped out without the program, 
the program is "saving” only 12 percent of all students for one additional year. Presumably, then, only 
those 12 percent could expect additional lifetime earnings because of the program. Third, the school 
provides the students with much more than the dropout prevention program. These other cosh; should 
somehow be considered. Finally, the usual charge that education contributes mm to a person’s life than 
increased earning power can certainly be leveled at this computation. 
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materials was in fact merely a deferred payment to purchase the EDL equipment 
already in the Rapid Learning Centers.) 41 The actual pilot project, summarized in 
Table 8, includes close to a third of the total expected enrollment in the two target 
schools involved (42 percent included at the junior high). This is large enough to 
have impact, but shows the drastic reduction in expectations (by being about the 
same-sized effort as last year in numbers of classes) from the full-scale introduction 
of turnkey reading and math into all secondary classes forecast a year ago. 



Table 8 

TRAINING AND SUPERVISION COMPONENTS : 
INDIVIDUALIZED INSTRUCTION 



Subj act 


College Hill 
Junior High 
Grades 7,8,9 


Arkansas 
Senior High 
Grades 10,11 


Total 


English 

Target: students 


349 


140 


4 89 


% of estimated 
enrollment aides 


3 


2 


5 


Mathematics 








Target students 


349 


140 


489 


% of estimated 
enrollment aides 


3 


2 


5 



Table 9 summarizes the four different Instruction Component programs. A sig- 
nificant change is that the RLCs are being operated by the school districts this year, 
in line with the recommendation made in the Final Evaluation Report. 48 Liberty- 



41 "Texarkana Turnkey: District Purchases $35,000 of EDL Materials,” Educational Marketer Vo! 
3, Nos. 21-22, August 1971, p. 7. 

42 Reducing objectives or hiring more staff was recommended in the 1970-71 Final Evaluation Report. 
Andrew and Roberts, op. eit., p. 118. . 
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Eylau is participating in the RLCs at full strength, but beyond that has only the 
ninth-grade vocational course as a new program. The Model Cities Reading Clinic 
supplements the contracted centers by taking in grades 1, 2, and 3, and will probably 
also support the Follow Through programs for the first grade. For the vocational 
orientation course, designing a course for the entire Washington sixth grade will 
clearly be the biggest challenge. The appearance of a vocational preparation course 
(at Arkansas High) in the proposal finally makes explicit the full dimensions of the 
career education effort that has been planned and supported informally for several 
years. 

Liberty-Eylau is rarely included among the other programs, which may reflect 
the realities of the extra work required by including another district. For example, 
the student information system and the PPBES system apply to the four Arkansas 
schools but not to Liberty-Eylau. Even more significant is their omission from the 
curriculum development effort. They participate in the counseling and special ser- 
vices efforts, but it clearly seems to have been important this year to try to minimize 
the problems of coordination and adminstration. 

The new project structure is given in Fig. 4. What is striking here is the heavy 
load on the manager of the curriculum and instruction component. Last year, cur- 
riculum and turnkey were combined unsuccessfully; this proposed combination 
seems if anything to be more demanding. Also of note are the number of contractors 
(five) with continuations on all but the PPBES from last year, and new personnel 
from the Region VIII ESC and from the Texarkana Mental Retardation Center 
added for the teacher training and supervision component. 



CONCLUSIONS 

The most notable achievement in the plans for next year is the attempt to treat 
in one program each of the likely contributors to the problem of students dropping 
out of school (before they have the skills or diploma to make it on their own). And 
none of the efforts except the PPBES and the peer tutoring are entirely new experi- 
ences, a definite management plus. From a larger view, the degree of coordination 
between this Title VIII program, other school programs, Model Cities, the regional 
Educational Service Center, and the Office of Education is remarkable. The most 
important example has been the integration of the dropout program with the educa- 
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Fig, 4 Project personnel organization chart 
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tion and manpower component of Model Cities (which supplied partial support), 
thereby providing a program with a strong chance of being relevant to the com- 
munity. 

It is especially promising to see the emphasis on training teachers and school 
personnel this year, since that was probably among the weakest operational areas 
last year (notably for turnkey personnel and the school administrators affected by 
the program). However, this does not affect the management and planning problems, 
and that area seems the most likely source of trouble, as in the past. As mentioned 
in the conclusions for last year, even though expertise is badly needed, bringing in 
a large number of contractors can create as many problems as it solves. Certainly, 
this and many other demands on management time can be greatly reduced by the 
simple expedient of pursui ng more Iiimtcu uuj ectives. The project director noted 
several times that bringing in a program from the outside requires an enormous 
amount of liaison work if it is to be at all acceptable to its recipients. As in other 
programs, there is the problem of involving the users in the detailed planning. This 
third year seems to have made a good start in that direction. Having the learning 
centers run by the schools should also generate more involvement, and allow greater 
flexibility of goals. 

Overall, the most important need seems to be to have explicit management 
training of key project and school personnel, going beyond what can be picked up 
by merely associating with a management support contractor. 



V. CONCI USIONS 



Performance contracting has been a mechanism for changing the educational 
process in the Texarkana schools. The Rapid Learning Centers and Laboratories and 
the turnkey classrooms that are designated as the second stage in the eventual 
dissemination of the new technology are different from the conventional Texarkana 
classrooms in materials and procedures. Nonetheless, the materials and procedures 
in the turnkey classrooms embody the 1969-70 RLCs’ learning system in only the 
most general fashion. Moreover, considering the expensive equipment used by EDL 
in the RLCs during 1970-71, and the complexity of its materials and procedures, it 
is hard to see how any inexpensive turnkey process could achieve much replication. 
Even if the EDL system hadbeen cost-effective as measured on an achievement-year 
basis, widespread replication of the system, or even introduction of some close 
modification of the system, would require two things: a considerably higher dollar 
expenditure per student, and greater logistics and managerial support. Considering 
the unhappy state of the Texarkana economy and the current low level of per- 
student expenditure, the first requirement does not seem likely to be met. And the 
second requirement would apparently be an obstacle, considering the difficulty dur- 
ing 1970-71 in providing materials for the turnkey classrooms on a timely basis, and 
the difficulty in providing training, monitoring, and assistance for the turnkey 
teachers. 

In terms of its prime goal of reducing the dropout rate, the dropout prevention 
program succeeded well in both 1969-70 and 1970-71; but equally, it failed to make 



much improvement in the academic achievement of its population of potential 
dropouts. And in each of its three years, it programmed goals far too ambitious for 
the available resources; nonetheless, the attempt to reach those goals has prepared 
the groundwork for such major innovations as program budgeting and district-wide 
manpower education programs. Thus, with all program elements controlled by the 
LEA in 1971-71, performance contracting seems to have been an effective catalyst 
for what was an inherently difficult process of change. 

In short, we expect that performance contracting will lead to a compensatory 
education program that is a substantial advance over the pre-1970 program in terms 
of the curriculum and instructional method. It may even lead to a curriculum 
improvement for Texarkana schools in general. When the turnkey process is com- 
pleted, however, we would expect the regular Texarkana classrooms to be using 
educational processes that only indirectly reflect the Rapid Learning Centers. 

Achievement measurement remains a vital problem. True, the definition of 
standards for inadmissible similarity between excercise questions and test questions 
and the better monitoring of materials have greatly lessened the chance of another 
test-teaching scandal. Nonetheless, serious problems remain. 

Some testing problems are logistic and administrative. It has been a strain to 
administer pre- and post-achievement Lis to a large group of students in addition 
to all the other diagnostic, interim-objective, and miscellaneous tests involved in the 
program. A more serious problem is the gap between the short-run objectives of 
instruction and what standardized norm-referenced tests actually measure. The 
nature of standardized norm-referenced tests makes them clumsy and inaccurate 
devices for measuring the effectiveness of a learning system. 

Texarkana in 1970-71 sought to deal with this problem by using criterion- 
referenced tests for part of the contractor’s payment and for evaluation purposes. 
This approach has considerable appeal, but it encountered difficulties. Constructing 
tests is a Herculean task. Administering them is time-consuming and creates many 
administrative headaches. In Texarkana the initial concept of testing all students 
and all objectives had to be replaced by a sampling procedure. The lesson would seem 
to be that the science and art of criterion-referenced testing still needs considerable 
development. 

The Texarkana evaluation procedures have been a very strong aspect of the 
program. Because the evaluator was located within commuting distance and be- 
cause his funding was unusually generous compared with that in some other per- 
formance contracting programs, he was able to provide almost continuous feedback 




to the program’s decisionmakers. This feedback has been highly useful and, at the 
time of the test-teaching scandal, almost essential. The use of an outside auditor 
presumably removes the chance that the close connection between the evaluator 
and the program management could lead to bias. 

In 1970-71, the link between the evaluator and the Texarkana program-man- 
agement was further strengthened by giving management-support duties to one of 
the former evaluators. We believe that the close linking between evaluation and 
management in this program is a model that could well be applied in other perform- 
ance contracting programs. There is, of course, the need to maintain public confi- 
dence in the announced program-outcomes; if there is a close link between manage- 
ment and evaluation, an outside audit may be important. We believe, however, that 
the prompt feedback of pertinent evaluation data is one of the most impressive 
features of the Texarkana program. 

Other features of the Texarkana program were less conducive to program im- 
provement, The proliferation of contracts and subprogram directors tended to 
diffuse and confuse lines of authority and responsibility. In addition, the perform- 
ance contracts for the RLCs monopolized the attention of the program director. In 
such a situation it would have been unreasonable to expect a program director to 
be able to assure that all parts of the program received their necessary share of 
management attention. 

The involvement of the Model Cities organization in getting the program going 
and providing funds at some crucial times was very important. Moreover, it provided 
the mechanism in 1970-71 for extending performance contracting into the elemen- 
tary schools, albeit on a modest scale. This history suggests the importance of some 
independent seed-money in the development or expansion of programs. 

Texarkana was the first major performance contracting program and received 
intense publicity. Since then, some of the glare of the spotlight has shifted to other 
performance contracting efforts, no doubt to the relief of Texarkana school officials. 
E-ven so, the 1970-71 experience demonstrates that Texarkana’s program still has 
many important implications for American education. 
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Appendix A 



CONTRACT BETWEEN TEXARKANA AND THE INSTITUTE 
FOR POLITICS AND PLANNING 



KNOW ALL tffiN BY THESE PRESENTS, that, Texarkana School 
District #7, Miller County, Arkansas, a public school system 
incorporated in the State of Arkansas with principal offices at 
Texarkana, Arkansas, designated as Fiscal Agent for a planned 
“dropout prevention project" to be funded by the U. S. Office of 
Education, hereinafter described as the "Agent", for and in con- 
sideration of five dollars ($5,0-0), and other valuable consider- 
ation, receipt whereof is hereby acknowledged, does hereby con- 
tract' and agree with the Institute for Politics and Planning, 
a corporation organigea under the laws of Washington, D, C», and 
with principal offices in Arlington, Virginia, but licensed to 




do business in the State of Arkansas hereinafter described as the 
"Contractor" as follows: 

WITNESSETH THAT: 

WHEREAS, the Agent has received a planning grant under 
the auspices of the U . S, Office of Education to plan and to 
conduct a "Dropout Prevention Program" in the Texarkana, USA, 
area, and desires certain technical and management assistance 
in the making of such plan, and 



technical assistance and advice to the Agent in the making of 
such plan, 

NOW THEREFORE, the parties do mutually agree as follows i 
1. Scope of Work - The Contractor shall, upon the 
terms and conditions herein set forth: 

(a) Analyse and determine the nature and extent 
of the "dropout" problem of the public school districts which 
will participate, hereinafter described as the "Participants", 
in the planned dropout prevention project, 

(b) Assist the participants to define and refine 
to operational terms the over-all objectives of the contractor- 
operated achievement center and work-study program, as indicated 



WIffiREAS, the Contractor is prepared to provide certain 
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in the Preliminary Proposal submitted to USOE on December 16, 

1968, hereinafter described as trie ''Preliminary proposal", 

(c) Assist the Participants to determine the 
criteria for selecting students who will participate and deter- 
mine the criteria for measuring the success of the program, 

(d) Assist the Participants to convert the objectives 
of the dropout prevention program to performance specifications 
and draft a mutually acceptable "request for proposals" which 
will be sent to potential contractors, after notification of 
award , 

(e) Assist the Participants to develop mutually 
acceptable criteria for evaluating sub-contractor proposals, 

{£) Develop and draft for approval an over-all 
operational program plan incorporating all the above and other 
requirements stipulated in USOE guidelines, as appropriate, 

(g) Develop and present to Participant for con- 
sideration of approval an over-all multi-year management plan 
which will include program budgets, costs and schedules, re- 
quirements for phasing "in" and “out" program elements, and 
requirements for project management assistance, 

(h) Assist the Participants to develop an over-all 
research and experimental design component. 



(i) Assist the Participants garner local , community, 
and other support for the effective implementation of the over- 
all project, 

(j) Draft a mutually acceptable final proposal to 
be submitted to USOE, 

2, Consultation - The Agent and Participants' staff 
members shall cooperate with the Contractor's representatives , 
and shall make themselves available at all reasonable times 
during ordinary working hours during the period of the contract 
to confer on any points which may arise as the project progresses 
The Contractor shall consult at least once every two (2) weeks 
with representatives of the Agent during the contract period. 

3, Period of Performance - The final proposal will be 
submitted for approval by the Fiscal Agent and others, as 
appropriate, and as soon as feasible but no later than May 1, 
1969, Professional assistance will be available for appropriate 
use until award of the, contract but no later than June 15, 1969, 

A preliminary draft proposal-report, written or oral, as appropri- 
ate, will be made no later than April 25, 1969, It is further 
understood that all of the above-stated dates are subject to 
change by mutual consent . 



4. Exceptions - In order to minimize costs and time, 
to the extent possible as mutually determined, maximum use will 
be made of local school officials, teachers, counselors, and 
other employees and employees of other local agencies which have 
had experience with or have knowledge about the nature of the 
dropout problem. When appropriate the contractor will request 
the administrative assistance of appropriate school officials 
and employees who have expertise in specific directly related 
areas , 

5, Consideration — The Agent shall pay to the Contractor 
a fee based on services rendered and direct costs incurred. The 
fee for professional services will be $100,00 per day for not 
mere than 60 days. Direct costs will include travel, subsistence, 
communications, clerical assistance, printing and other reason- 
able expenses incurred in the performance of the contract. The 
Contractor shall be paid at the end of each thirty (30) day 
period, commencing thirty (30) days from the date hereof or 
letter of intent to enter into contract proving that funding 
agency authorization has been awarded to the Agent, A detailed 
invoice showing each item of services and expenses shall be 
provided to the Agent by the Contractor for each payment, and 
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the Agent shall not be liable for any payment until such invoice 
is furnished. Final payment under the contract shall be made 
not less than fifteen (15) days after submission of Contractor's 
final report and invoice. Total payment will not exceed 
$9650.00 (nine thousand six hundred fifty dollars), 

6. Contractor Staff - Contractor's "Project Director" 
for this contract shall be Mr, Charles Blaschke, assisted by 

Dr, Joseph Hart, and two other qualified consultants as required, 

7 , Overtime - No overtime compensation shall be paid to 
the Contractor for any work performed hereunder, 

8, Availability of Office space - The Agency shall 
provide such temporary office space as may be necessary for use 
of professional staff members of Contractor in the Texarkana, USA, 
area during the period of the contract . 

9. inspection and Interim Reports - The Agency shall h^ve 
-the right at all times during the period of the contract to 

inspect the work performed by the Contractor, and to request 
brief interim oral or written reports of work progress from 
the Contractor as may be reasonably necessary to assure proper 
performance ctf the Contract . 
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10, copyrights and Reproduction - Reports, surveys, and 
3 ther information produced under this contract shall be the sole 
property of the Agent, including copyrights and proceeds from the 
sale or reproduction thereof, if any, except that the Contractor 
shall liKewise have the right to reproduce and use such reports, 
surveys, and other information as it produces under the contract, 
provided any income or profit arising therefrom, if any, shall 

accrue to the benefit of the Agency, 

11. Changes and Conditions - Changes, additions or 
conditions to this contract may be made only by mutual agreement 

of the parties , 

IN WITNESS WIDSREOF, the parties hereto have executed this 
contract this / day of March, 1969. 





APPROVED BY: 



Fiscal Agent 

The Institute for Politics and 
planninc 




President 




ESTIMATED BUDGET 



! 

f 

| 

! 



1, Technical arvices 60 man 

2, Travel (Blaschke and Hart) 

(Staff consultants) 

3, Telephone 

4, Administrative-Clerical 

5, Subsistence 



days at $100,00 $ 6,000,00 

1 , 200.00 

400,00 

400.00 

1 , 000.00 

450.00 



6, Printing and reproduction 

7, Expendable Supplies 



50.00 

150.00 

Total $ 9,650.00 
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Appendix B 

TEXARKANA-BORSETT CONTRACT 



SUBCONTRACT BETWEEN THE LEA FOR THE 
TEXARKANA DROPOUT PREVENTION PROGRAM AND 
DORSETT EDUCATIONAL SYSTEMS, INC. 



Purpose 

This subcontract is based upon the rfp dated 6-10-69, issued by 
the LEA, the proposal submitted by Dorsett, and a mutually agreed 
upon Letter of Intent. It is intended to stipulate the scope of work, 
responsibilities, and obligations assumed by both parties, but to the 
extent that further details are required to interpret matters aris- 
ing under it the above documents are incorporated by reference. 

I, Period of Contractual Obligation 

The period of contractual obligation begins September 10, 1969 and 
extends until June 5, 1970. 

II. Previous Obligation 

The grant terms and conditions of grant § QEG-0-9-130045-3360 Project 
it 13-0045 between LEA, Texarkana, Arkansas School District # 7 and 
the U. S. Office of Education are incorporated herein by reference 
and made a part of this contract. 

IH . ' General Scope of Work Assumed by Dorsett 

Dorsett agrees: 

a. to organise and operate the^instruetional component of the first 
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phane of the Texarkana Dropout Prevention Program, 

u, to provide instruction in basic reading, math and study skills 

to a minimum of 200 student:) . The study skills may be measured by 

inference of the achievement in math and reading areas, 

c, to hire and train local personnel, if possible these people will 

come from the target area, as para-prof essienais in the operation of 

the instructional program, 

d» to utilize at least 20 teachers and administrators from thepartlci- 
patlng school systems who will work part-time In th© instructional 
program and will facilitate ths comtemplnted transfer of the Doraett 
material to the Texarkana Rapid Learning Centers , Their first hand 
knowledge of the nature and extent of academic problems unique to the 
Texarkana schools will he useful to the contractor, 
e, to operate centers at locations mutually agreeable to the parties, 

IV. SE LECTION OF STUDENTS 

a. All students who participate in this instructional program will 
have grade level deficiencies, in reading and math, of 2.0 or more 
as determined by the Iowa Test of Basle Skills or the SRA Testa, 
Further, all of these students will have no less than the minimum 
Intelligence Quotient, as determined by Lorge Thorndike ana IRA Ability 
Quotient, of a regularly enrolled student as required by the two 
school districts, seventy in Texas and seventy-five In Arkansas, by 
the Project .Management Office or its delegated representative. 

b. All students who participate in the first phase of this instruct- 
ional program will come from grades 7-12 in the regular school system. 

c. The makeup of the first 200 students will consist of approximately 
equal numbers of volunteers, students assigned by counselors, and 
students randomly selected from those with a grade level deficiency 
of 2,0 or more. 

d. The makeup of any group of students beyond the initial 200 will be 
similar to that of the first 200, or will have characteristics deter- 
mined by the LEA and stipulated by the reference material. * (RFP, 
Dorsett *s proposal, and the Letter of Intent). 



a. The entry status for each student will be determined by the most 
recent test. The Texarkana Arkansas school system used 1TBS Form 3 
and the Liberty Eylau school district used SRA Achievement Series 
Form D. These tests were given the first week of October, 1969. In all 
eases the tests were given on a group basis and the counselors in the 
individual schools administered the tests. The same conditions will 
exist for the post-test as was the case in the pre-test, 

b. The parties agree that Dorsett will have the option to ask for 
retesting or adjustment to entry level standing determined by pre- 
tests where Its diagnostic test shows a substantial difference and 
that the pre-teat may have been Insensitive to the actual grade levs! 
deficiency when the deficiency is 2.0 grade levels or more. Diag- 
nostic test given by Dorsett should be administered under conditions 
similar to that of the initial pre-test. Further, Dorsett will not- 
ify the LEA as to what diagnostic test will be used and will allow 
observation of the testing by theProjeet Manager or the Internal 
Evaluator. The negotiation of the interpretation of these tests will 
be handled by Dorsett ’s representative and the Project Manager with 



V. Testing 
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b- The student' performance differential is determined by subtracting 
the entering grads level achievement in math and reading from the 
exit level. Entry status and exit status are based on the SEA and 
ITBS tests as weighted on a basis to be detern;‘ned no later than 
February 1, 1970. This procedure will be applied to all assignees 
except withdrawals, and a small number of students, assigned by non- 
random procedures , to be mutually agreed by the parties to this 
contract , for whose learning services Dorsett will be reimburs d at 
the average hourly rate of other students. 

c . Dorsett will be compensated on the basis of obtaining one grade 
level increase per subject area in eighty hours of instructional 
center study for $80.00, or proportionally for each fraction thereof 
For students requiring more or less than 80 hours per subject grade 
level increase, the payment to Dorsett per subject grade level Incr- 
ease will vary according to the formula $80.00 x 80 hours divided 
by actual study hours required per subject grade level increase. 
According to this formula, one grade level Increase per subject area 
In 110 hours of Instruction would cost $58.18, Both parties agree 

hat $ 106.67 for 60 hours represents the upper limit of the cost 

.jimbursement formula and that if over 110 hours of instruction are 
required, the payment for a grade level increase will be reduced 
by $1.00 per hour for every hour over 110. This payment schedule 
will result in no payment to the contractor if 168 or more hours are 
required for one grade level achievement , 

d, Monthly progress payments may be made to Dorsett for reimburse- 
ment of not more than an estimated 85 % of direct and indirect costs 
Incurred by Dorsett for its 1 operations, provided further that the 
payments do not exceed the estimated accruals to Dorsett for grade 
level gains, based on sampling tests or progress check tests, in the 
professional Judgement of the Project Director, It is noted that 
repeated testing with the same or similar test Instruments used for 
final audit on student dlsassigmnent would contaminate the validity 
of results j so different tests must be used for interim evaluation. 

IX, Availability and Cost of Capital Equipment 

a, Dorsett agrees to sell 95 units of the Dorsett M 86 Teaching Mach- 
ines at a unit price of $200.00 for a total of $19,000.00, All 
equipment will carry standard warranty. In the event that the con- 
tractor fails to achieve substantial gains in the program Dorsett 
will repurchase the equipment at full price 

b. During the period of this contract, Dorsett is responsible for 
the full maintenance and upkeep of the Dorsett manufactured equip- 
ment. In accordance to the standard one year warranty, repairs 
will be made on a 24 hours basis or another M86 machine will take 
its place. An adequate amount of supplies and parts for the M86 
will b# available. The training of local personnel for maintenance 
of the M86 will also be part of the program. 

X. Us e of Consultants Listed In the Dorsett Proposal 

It Is understeed that all key consultants or persons of similar 
status and staff members listed in the Contractor Proposal will be 
used or, a workinc level , including site visits. Deletion or add- 
.itj.'-'-.i of consultants must be mutually agreed upon by both parties. 

■■■ t b, .atisfied tii to the active participation of those 
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X I * Availa bilit y _gf Instru c tional Materials 

a. Materials to be used in this instructional program will substan- 
tially duplicate that listed in the Dorset! Proposal. . 

b. Dorsett wilJ provide materials for ned i um-sn-d --high' achieving stu- 
dents and will have such material available at the instructional 
centers for testing with a sample population no later than April 30, 
1970. 

XU. C ommun i ty and Public Relations 

a, ?ne LEA is responsible for informing parents, Instructional cen- 
ter employees, and students about testing procedures, scheduling, 
dismissal, and progress reports. 

b. All official press releases concerning this program should orig- 
inate from LEA, 

XIII Review of Contract 

The parties agree that from time to time the LEA may 4 * view 
progress on the program and ask for contract amendments if reasonably 
anticipated progress is not being obtained, 

XIV, Applicable Statute s 

In case of conflict arising under this contract the laws of the 
State of Arkansas will prevail. Unless otherwise stipulated, parties 
will be bound by the request for proposal and the proposal of the 
Contractor, 

XV, Officials Not to Benefit 

No member of or delegate to Congress, or resident eoi-iilssionsr , 
shall be admitted to any share or part of this contract, or to any 
benefit that may arise therefrom; but this provision shall not be 
construed to extend to this contract if made with a corporation for 
its general benefit, 

XVI , Covenant Against Contingent Fees 

The Contractor warrants that no person or selling agency has 
been employed or retained to solicit or secure this contract upon an 
agreement or understanding for a commission, percentage, brokerage, 
or contingent fee, excepting bona fide employees or bona fide estab- 
lished commercial or selling agencies maintained by the Contractor 
for the purpose of securing business * For breach or violation of 
this warranty the Fiscal Agent shall have the right to annul this 
contract without liability or in its discretion to deduct from the 
contract price or consideration, or otherwise recover, the full am- 
ount of such commission, percentage, brokerage, or contingent fee, 

XVII, Equal Employment Opportunity 

(Section 202, Executive Order 11246, September 24, 1965 * 30 FR 11269) 
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"During the performance of this contract the contractor agrees as 
follows; " 

"(1) The contractor will not discriminate against any employee 
or applicant for employment because of race, creed, color, or national 
origin. The contractor will take affirmative action to ensure that 
applicants are employed, and that employees are treated during onr* 
ployment , without regard no their race, creed, color, or national 
origin* Such action sh 11 include, but not be limited to the follow- 
In g: employment, upgrading, demotion, or transfer, recruitment ad- 

vertising; layoff or termination; rates of pay or other forms of com- 
pensation; and selection for training, including apprenticeship* The 
contractor agrees to post in conspicuous places, available to em- 
ployees and applicants for employment, notices to be provided by the 
contracting officer setting forth the provisions of this nondiscrim- 
ination clause * 

» (2) The contractor will, in all solicitations or advertisements 
for employees placed by or on behalf of the contractor , state that 
all qualified applicants will receive consideration for employment 
without regard to race, creed color, or national origin, 

"(3) The contractor will send to each labor union or represen- 
tative of workers with which he has a collective bargaining agree- 
ment or other contract or understanding, a notice, to be provided by 
the agency contracting officer advising the labor union or workers_ 
representative of the contractor's commitments under Section 202 oi 
Executive Order No * 11246 of September 24, 1965# and shall post copies 
of ‘the notice In conspicuous places available to employees and app^ 
lleants for employment, 

•* (4 ) The contractor will comply with all provisions of Executive 
Order Ho, 11246 of September 24, 1965, and of the rules, regulations, 
and relevant orders of the Secretary of Labor, 

*‘(5) The contractor will furnish all information and reports re- 
quired by Executive Order No, 11246 of September 24, 1965, and by the 
rules, regulations, and orders of the Secretary of Labor, or pursuant 
thereto, and will permit access to his books, records, and accounts 
by the contracting agency and the Secretary of Labor for purposes of 
investigation to ascertain compliance with such rules, regulations 
and orders, 

" (6 ) In the event of the contractor's noncompliance with the 
nondiscrimination clauses of this contract or with any .of such rules 
regulations, or orders, this contract may be cancelled , terminated 
or suspended in wh©3e or in part and the contractor may be declared 
ineligible for further government contracts in accordance with pro- 
eedures^authorized in Executive Order no. 11246 of September 24,1965 
and such other sanctions may be imposed and remedies invoked as pro- 
vided in Executive Order No, 11246 of September 24 s 1965, or by rule, 
regulation, or order of the Secretary of Labor, or as otherwise pro- 
vided by law, 

! '(7) The contractor will include the provisions of Paragraphs (1) 
through (7) in every subcontract or purchase order unless exempted 
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by rules , regulations, or orders of the Secretary of Labor Issued 
pursuant to Section 20t of Executive Order Ho* 112b 6 of September 2b, 
196: /"so that such provisions will be binding upon each subcontract 
or vendor* The contractor will take such action with respect to any 
subcontract or purchase order as the contracting agency may direct as 
a means of enforcing such provisions Including sanctions for noncom- 
pliance: Provided, however , that in the event the contractor becomes 

involved in, or is threatened with, litigation with a subcontractor 
or vendor as a result of such direction by the contracting agency, the 
contractor may request the United States to enter into such litigation 
to protect the interest of the United States* 11 

XVIII* Certification of Non-Segregated Facilities 



The contractor or subcontractor certifies that he does not main- 
tain or provide for his employees any segregated facilities at any 
of his establishments, and that he does not permit his employees to 
perform their services at any location, under his control* where 
segregated facilities are maintained* He certifies further that he 
will not maintain or provide for his employees any segregated fac- 
ilities at any of his establishments, and that he will not permit his 
employees to perform their services at any location under his control, 
where segregated facilities are maintained. The contractor or sub- 
contractor agrees that a breach of this certification is a violation 
of the Equal Opportunity clause in this contract. As used in this 
certification, the terra ^segregated facilities" means any waiting 
rooms , work areas, rest rooms and wash rooms, restaurants and other 
eating areas, time clocks, locker rooms and other storage or dressing 
areas, parking lots, drinking fountains, recreation or entertainment 
areas, transportation, and housing facilities provided for employees 
which are segregated by explicit directive or are in fact segregated 
on the basis of race, creed, color, or national origin, because of 
habit , local custom, or otherwise# He further agrees that (except 
wh*-re he has obtained identical certifications from proposed sub- 
contractor; for specific time periods; h: will obtain identical cer- 
tifications from proposed subcontractors prior to the award of ^sub- 
contracts exceeding $10, jIO.00 which are not exempt from the provis- 
ions of the rknjc. 1 Opportunity Clause* that he will retain such cer- 
tification;: iu his riJ.orf; and t.iib re w.iI2 forward the following 
notice sueVi proposed subcontractors (except where the proposed sub- 
contractors Have submitted identical certifications for specific 
time periods): 

XIX, Notice to Prospective Subcontractors of Requirement for Certi- 
fication:? of Nc ■segregated Facilities 

A Certification of Nonsegregeted Facilities, as required by the 
May 9, 1967, order (32 ?.R. 7^39, May 19, 1967) on Elimination of 
Segregated Facilities, ty the Secretary of Labor, must he submitted 
prior to the award of a subcontract exceeding $10,000.00 which is^ 
not exempt from tha provision;? of the Equal Opportunity clause, ihe 
certification may submit ted either for each subcontract or ,or ail 
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i .mually ) • 
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ly , semiannually , or 



Note; The penalty for making false statements in offers in prescribed 
in 18 U.S.C 1001/ 






y- _ * 

7—J. iAidL-^ z 

Edward D. Tr^ce, Fiscal Agent 
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Appendix C 

LETTER OF INTENT, TEXARKANA TO DORSETT 



Texarkana School piotrict no. 7 

orrice or THU oonisiiiNTcNDCHT 
TEXARKANA. ARKANSAS 
September 12, 1969 



Loyd Dorsett, President 
Dorsett Educational Systems, Ine. 

Norman, Oklahoma 

Dear Mr. Dorsett j 

This Letter of Intent is based upon the Request for Proposal 
issued by the fiscal agent and the Proposal submitted by Dorsett 
Educational Systems, Inc,, both of which are Included by reference. 

It is intended to outline the mutually agreed upon provisions which 
are stated in the Proposal and which will be the basis for a contract 
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to be prepared and executed by the Fiscal Agent and Dorsett Educational 
Systems* 

Dornett has agreed to organize ana operate the centers and to be 
compensated on the basis of obtaining one grade level increase in eighty 
hours of center study for $30.00 as stated in the proposal with limits 
on the formula .matrix being set as follows* the upper limit is eighty 
hours for approximately $107 *00; the lower limit is to be set between 
100-110 hours with the precise limit being negotiated between the parties* 

It is mutually agreed that the makeup of the first 200 students 
will consist of approximately 50 volunteers," 50 students assigned by 
counselors t and the remaining 100 randomly selected from those with 
grade level deficiencies of 1*5 or more* Based on this computation, 
Dorsett will grant a 7% discount on the payment per grade level increase s 
if the average I*Q, of the 50% of students randomly selected is 100 or 
greater. Procedure for selection and the termination of discount for 
additional assignees will be mutually agreed upon. 

Both parties agree that in addition to the regular vocabulary test 
used to determine student level, differential vocabulary tenting developed 
in conjunction with local teachers may be used with the weight given to 
this test to be determined by the local school boards. 

The Project Manager will have the option to retest the enrollees 
for retention, at any time up to six months after leaving the center, 

£©!f comparison of retention to normal achievers in the local area 
which could be a basis, for adjustment. 

The entry status will be determined by the most recent Iowa Tests 
or equal , and if tested prior to August, 1969, but no more than one 
year ago, the entry grade level will be adjusted upward based on the 
usual Growth Curves, 

Dorsett will have access to the utilities, bell, and intercom 
services of the school being serviced where a mobile facility is moved 
onto the school grounds; and Dorsett will not be reimbursed for the 
cost of the mobile center to be used at the Arkansas High School. 

Hater inis listed in the proposal will be purchased through the local 
school system for benefits of delivery and discounts* Dorsett will 
assist the local school purchasing agent in acceleration of the pro- 
curement , 

The Parties agree that Dorsett will have the option t© ask for 
retesting or adjustment to entry grade level standing determined by 
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pre-tests where a diagnostic test shows a substantial difference and 
that the pre-test may have been insensitive to the actual grade level 
deficiency when the deficiency is 2.0 grade levels or more. 

Dorsett may use one or two teachers on a sabbatical basis ©n 
approval of the local school board with the local school system being 
reimbursed from contractor payments for teacher salaries. 

The Parties agree that the Letter of Intent is being used to allow 
Dorsett to begin work so the center or centers will be in operation, 
subject to refurbishment and purchasing problems , by October IS, 1969. 

The Parties agree that after 120 days of operations, the Project 
Manager may review progress on the program and ask for contract amend- 
ments if reasonably anticipated progress is nor. being obtained. 

The Parties agree that the $80/80 hour formula will be slightly 
higher for the first 200 studentSjand will be decreased as the operations 
are extended to additional students based on operational efficiency 
reflecting economies of scale* 

The Parties agree that materials for medium and high achievers will 
be available at the centcr ; and will be tested with sample population no 
later than April 30, 1970. 

Dorsett has agreed to quote prices on its equipment utilized in 
the center, for future purchase up to September f 1973* 

It Is understood that the consultants and staff members listed 
in the proposal will be used on a working level, including site visits. 



IDTsra 



Yours truly 



,7rz) rVi 



= V/ , 

Edward D, Trice 




Fiscal Agent 

Rapid Learning Center 
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Appendix D 



CONTRACT BETWEEN TEXARKANA AND REGION VIII 
EDUCATION SERVICE CENTER, 1969-70 



REGION VIII EDUCATION SERVICE CENTER 
P. O. BOX 689 
Magnolia, Arkansas 71753 



This contract is made between the Region VIII Education Service 
Center (Magnolia School District No. 14 ) , hereinafter known as 
the contractor and Texarkana Dropout Prevention Program (Texarkana 
School District) , hereinafter known as tha purchaser . 

The contractor agrees to provide and the purchaser agrees to 
request end accept on a frequent and continuing basli, tha intamal 
avaluation of the Texarkana Dropout Prevention Program. This 
service is to be provided beyond those normally provided from 
Title III, ESBA f\mds during the 1969-70 school year. 

for the internal evaluation services, the purchaser agrees to 
pay the contractor the total amount of $11,500.00, with one fourth 
of this e^ount ($2,875.00) to be paid on December 30, 1969 and 
one fourth at each of the following dates t February 28, 1970; 

April 30, 1970; June 30, 1970, at ,^pletion of the evaluation 
report. 



SERVICE CONTRACT 
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The proposed expenses are established as fellows * 



Special Consultants $2,000 
Travel 800 
Clerical k Material 1,000 
Data Processing 500 
Region VIII Personnel 7,200 
TOTAL fllvtW 



This contract may be renewed annually by mutual consent of both 
contractor and purchaser and is signed in triplicate by the 
following official representatives of the contractor and 



purchaser. 






% «-;CdhL3a' # ""'i? iscal Agent" 

Texarkpja Dropout Prevention Program 

.. ... . , — — 
carl .,.3 u ualcy , ^^perintendent 

Kstgnolia School District Ho. 14 






7 




Date Sighed 



/? - /a 



fraa <■/ .> A fvw3k.. 

Fcywli, .,»nTbi. D., Dire 



'Director 

Region VSXZ Education Servlet Center 
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CONTRACT BETWEEN TEXARKANA AND EPIC 
EVALUATION CENTER, 1969-70 



SUBCONTRACT BETWEEN THE LEA FOR THE 
TEXARKANA DROPOUT PREVENTION PROGRAM AND 
EPIC EVALUATION CENTER 



This subcontract is intended to stipulate the scope of work, respon- 
sibilities, and obligations assumed by both parties. The purpose of 
the Educational Audit is to verify the activities and results reported 
by the internal evaluation of a project or the instructional program 
of a local educational agency. Since the basic purpose of the aval** 
uation program is "to provide information for decision-making," the 
educational audit should provide verification of information regarding 
three basic aspects of instructional programs I 

1. Program management 

2. Program effectiveness 

3. Program costs 

I. Period of Contractual Obligation 

The period of contractual obligation begins December 1, 1969 and ex- 
tends until June 5, 1970. 4 

II. Previous Obligation 



The grant terms and conditions of grant //OEG-9-130Q45-3360 Project 



#13^0045 between LEA, Texarkana Arkansas School District #7 and the 
United States Office of Education are incorporated herein by reference 
and made a part of this contract. 

Ill , S cope 

The scope of the educational audit is limited in a general way to the 
process of the Internal evaluation program. However, should the in- 
formation from the internal evaluation program be insufficient in 
providing information for dtn. y! on-making* the scope of the educational 
audit should include suggestions and recommendations to these concerns. 

IV* Reports 

Tha general responsibility of the educational audit is to check, verify 
and report on the information provided by the internal evaluation pro^ 
gram* To be as meaningful as possible to the project directors three 
main reports should be made* 

1, A Planning Report— -this report is typically concerning the 
scope and adequacy of the planned evaluation program at the 
beginning of the project operation on or before December 1, 
1969 * 

2* A Process Report—this report deals with the implementation 
process of the planned evaluation program, particularly the 
monitoring systems and schedules of evants on or before 
February 1969, 

3, A Recycling Report^^thls final report reviews the findings, 
results, and recommendations of the internal evaluation pro- 
gram with regard to program management, 

V. Coots 

The EPIC Evaluation Center, Tucson, Arizona, agrees to fulfill the 
function of educational auditor for the Dropout Prevention Program, 
Texarkana^ Arkansas, for the total amount of $5,111*70, with two^ 
thirds of this amount ( y 3 ,407, 80) to be paid to the EPIC Evaluation 
Center in advance, and ©ne^third of the amount ($1,703,90) to be 
paid at the completion of the final audit. 

The proposed expenses are established as follows t 
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Travel $ 632.L 0 

Per Diem 240.00 

Clerical, Materials 800.00 

Computer facility 475,00 

Personnel 2,500.00 

4,647.00 

Administrative Overhea d' 464.70 
Total $ 5,111.70 



Robert E. Kraner, Assistant Director LEA 

EPIC Evaluation Center Texarkana Dropout Prevention 

Program 



AGREEMENT 



*the EPIC 1 aluatlon Center, 1034 East Adams, Tucson, Arizona, agrees to 
arform the Junctions and responsibilities of the outside educational accomplishment 
auditor for the Texarkana Dropout Prevention Program, Texarkana School D.atri<;t, 
Texarkana, Arkansas, for the consideration of three thousand, four honored eightf- 

«idit dollars and ten cents 063,488. 10). 

The primary responsibilities of the auditor wfll be to; 

a. verify the results of the project evaluation, and 

f). assess the appropriateness of the evaluation procedures. 



i. S ervices and Products 

The audit plan is referenced to the Texarkana Dropout Prevention Program, 
Texarkana, Arkansas, grant #0130-9-130045-3360, Project ,13-OMS. Services to 
be provided are* 



a. to critique the evaluation plans submitted by the Internal evaluator 
for all project components and to make general recommendations 
regarding their effectiveness. 

b. to critique, verify, and make general recommendations with re- 
gard to the products and processes of the internal evaluator (see 
Appendix A), These will include the following;-* 

(1) Identidcatiorr of pertinent variables 

(2) Behavioral Objectives 

(3) Adequacy of measuring instruments 

(4) Monitoring systems 

(5) Calendar of events 

c. To provide two audit reports to the LEA In accordance with para- 
graphs 4 and 6 hereof. These two reports — Into, 'm Report I and 
Interim Report U — will be based upon information gathered from 
project records, interviews with pn. ’eel personnel, and data gathered 
from specified measuring instruments utilized by the internal evalu- 
ator. A minimum of four on-site visits wUi be made by an EPIC 
representative and three progress cheeks will bo made to the proj- 
ect director during the time of this contract, 

2, Audit Personnel 

Dr. Robert E, Kraner, Assistant Director, EPIC Evaluation Center, will 

serve as Project Audit Director, utilizing tee specialties of the EPIC staff in the 

required areas. The qualifications of these personnel are given in Appendix B., Any 

changes in the assigned staff will be contingent upon approval of the Project Director 

and U30E representative. The anticipated amount of time required for the audit tone 

tion by the Project Audit Director and specialties of the EPIC staff are as follows; 

Project Audit Director, Dr. Robert Kraner U days 

Project Advisor, Management Techniques, Dr. Robert 

*T, Armstrong 2 days 

Project Advisor, Research and Statistics, Dr, Terry 

D. Cornell 2 days 
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Project Advisor, Low Achieving Students, Dr, Richard 

H, Powell 2 days 

3, Requirements for Space and Documents 

EPIC has no need for permanent facilities or secretarial assistance within 
the Dropout Project; however, * it is expected that suitable temporary facilities will 
be available during on-site visitations', and that transportation will be provided dur- 
ing on-site, visits between facilities, 

EPIC will require the following documents be provided during the, initial 
audit activities: 

a, USOE Guidelines governing the project 
K Complete and corrected copy of the project proposal 
e. Copy of pertinent correspondence and publicity releases 

d. copy of all sub-contracts of project 

e. Actual budget expenditures 

f. Measurem"nt Instrument for each stated behavioral objective 

4. Schedule of Reports 

it is the intent of the EPIC Evaluation Center to review as completely as 
possible the activities of the internal evaluavr of the project. The results of these 
reviews will be presented in two main written reports during the time of this con- 
tract: 

a. Interim Report I March 15, 1970 

b. Interim Report II April 1 , 1970 

The content aid scope of these major audit reports will be entirely dependent 
upon the written report of the internal evaluator for the project, 

6, Sampling Techniques 

All forms, checklists, and tests used in the project fay the Intend evalu- 
ator will be evaluated as to validity and reliability by testing specialists -at tho EPIC 
Evaluation Center, The qualifications of test administrators, testing procedures, 
test scoring, and analysis of results will be verified. 

i ftd 
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Due bp the importance of the achievement test data for use in payment of 
project funds, all achievement testing techniques and scoring will bo spot-checked 
and the analysis e f results wlU be r ©-calculated at the Center, These results will 
be -made available to the Project Director and will be included in th© Final Audit 
Report, 

6, Audit Reports 

EPIC will hold periodic progress checks with the Project Director to verify 

4 

the reports of the internal evaluator. All written imports will go directly to the 
Project Director, Fifty copies of thp Final Audit Report will be delivered to the 
Project Director, 

The Final Audit Report will include verification of all findings and bond - • 
slons submitted in writing by the internal evaluator and the assessment of the ap- 
propriateness of evaluation procedures, 

7, Confidentiality 

Only those documents outlined hi - Paragraph 3 of this contract will be re- 
quested from the project. All information and findings will be held in strictest 
confidence by EPIC, 



8, Payment Schedule 

The EPIC Evaluation Center shall be entitled to a fixed payment in the 
amount of three thousand, four hundred eighty-eight dollars <$3,488,00), with 
one-half of this amount $1,744,00) to be paid upon receipt of Interim Report H, 
April 1, 1970, and the remaining one-half ($1,744,00) to be paid upon receipt of 
the proposed audit contract for 1970-71, Jime 4, 1970, 

9, Grant Terms and Conditions 



Any publicity release must have the approval of the LEA, 



the Grant Terms and Conditiona of Grant #OEG-9-l3OO45 i, 330O ( Project 







#13-0045 between the Texarkana Public Schools and the U,S, Office of Education 
are made a part of this agreement. The obligations of this agreement shall begin 
at 12:00 p, m, , EST, March 10, 1970, and shall terminate at 12:00 p. m. , EST, 
June 6, 1970, with subsequent work to be done by EPIC under a new agreement,, 

10, Entire Agreement 

This contract constihites the entire and only agreement between the parties 
named hereto and may be ammended by an instrument in writing by authorized sig- 
natures and the date tt roof with the intent to be bound thereby. 



EPIC Evaluation Center 
Tucson, Arizona 



by . <^? /^w 

Robert Kraner, Assistant 



si slant Director 



date: 3 — 6 ~ ~ 7 & 



Texarkana Public Schools 
Texarkana, Arkansas 




b 
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Appendix F 

TEX ARK AN A-EDL CONTRACT 



CONTRACT 
BETWEEN THE 

TEXARKANA SCHOOL DISTRICT it 7 
and 

EDUCATIONAL DEVELOPMENTAL LABORATORIES, INC* 
A DIVISION OF McGRAW-HILL 



THIS CONTRACT s made and entered into this 18th day of September* 
1970, by and between the Texarkana School District #7, a public school 
District organised and existing under the laws of the State of Arkansas, 
with principal offices located at 1500 Jefferson Avenue, Texarkana, 
Arkansas 75501 (hereinafter called LEA), and the JOINT VENTURE comprised 
of EDUCATIONAL DEVELOPMENTAL LABORATORIES, INC*, a Division of McGraw- 
Hill, a private corporation organized and existing under the laws of 
the State of New York with principal offices located in Huntington, 

New York, (hereinafter referred to as the Contractor), and Arkansas 
School Service, Inc* , a private corporation (a franchised dealer of 
EDL /McGraw-Hill) organized and existing under the laws of the State 
of Arkansas with principal offices located at 1911 Thayer Street, F*0. 
Box 2901, Little Rock, Arkansas 72203, and Texas Educational Aids, a 
private corporation (a franchised dealer of EBL /McGraw-Hill) organized 
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and existing under the laws of the State of Texas with principal offices 
located at 120 East Elm, Tyler, Texas 75701. This contract is based 
upon the Texarkana School District #7, Arkansas, RPP #2 and the contin- 
uation proposal financed by tLE. Office of Education administered ESEA 
Title VIII grant number OEO-0-9- 130045-3300 281) , the Proposal submitted 
by EDL August 13, 1970, and Addendum September 15, 1970, and documented 
negotiated details September 24, 1970, and is incorporated by reference 
and made part, hereof* 

It is intended to stipulate the scope of work, responsibilities, 
and obligations assumed by both parties. If further details are re- 
quired to interpret matters arising under it, the above documents and 
all controlling local state, and federal laws and regulations and their 
issues are incorporated in this contract by reference. In instances of 
conflicts within and between said incorporated aments, resolution 
will follow, in descending order of authority : (1) Federal laws, reg- 
ulations, and t *ir issues; (2) State laws, regulations, their 

issues; (3) Local laws, regulations, and their issues and (4) Mutual 
convenience of the contractual parties. 

Perf G a nance under this contract shall commence September 28, 1970 
and terminate June 30, 1971. 

OPTION TO RENEW 

A, By April 1, 1971 the Contractor will submit .. six copies a de- 
tailed statement of work planned to be accomplished during the 
next program year and six copies of a detailed P.P.B.5. budget 
to support this plan, 

B- The LEA will provide written notice to the Contractor by June 21, 
1971, based on the meeting and agreement reached by the combined 
school boards at their June 15, 1971, meeting of their option to 
review the program for the subsequent year. 

I, SCOPE OF WORK 

The long-range goals of the Texarkana Dropout Prevention Program 
are i 

1. To significantly reduce the percentage of dropouts in the 
Texarkana and Liberty-Eylau school districts, 

2. To increase academic achievement and skill development of 
students who are educationally deficient, 

3. To increase the cost effectiveness of the instructional 
program in the Texarkana and Liberty-Eylau school districts. 
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IX, DUTIES OF CONTRACTOR 

Using the existing facilities, the Contractor shall Establish and 
operate a teacher support program at a minimum of one learning 
center located at each of the following schools: College Hill 

Junior High School; Jefferson Avenue Junior High School; Arkansas 
Senior High School; Lib erty-Eylau Junior High School; anrf Liberty- 
Eylau Senior High School, 

III, RESPONSIBILITIES OF CONTRACTOR 

1, The Contractor agrees to provide an instructional learning 
system appropriate to the individual needs of the target 
population* 



2. Whenever appropriate* the Contractor agrees make maximum 
use of LEA facilities and equipment resource located at the 
school sites, i,e*, mobile units, furnishings* desks, etc. 

3, The Contractor agrees to purchase, assemble, install, and 
maintain all Contractor-owned equipment which will be utilized 
during the school year at his costs, 

4* The Contractor agrees to apply all rental costs to the pur- 
chase of any equipment and material on lease at the price 
quoted in the Contractor f s 1971 published catalog. The LEA 
will have the option to exercise its rights under this con* 
tract at any time prior to June 30, 1971 , for all equipment 
and materials used during the 1970-1971 school year* The 
Contractor agrees to conduct program operations for students 
in the late afternoon or early evening* The additional cost 
to LEA for operating these evening centers shall not exceed 
the established costs for the operation of regular learning 
centers for similar students, 

5. The Contractor agrees to conduct his operational program 
within », contraints of, and in accordance with, the in- 
tent and conditions of the evaluation design, 

6. The Contractor agrees to obtain the approval of the LEA in 
■employing all instructional personnel used in the project . 
Whenever possible, personnel will be employed from the local 
community * 

7. The Contractor agrees to train and monitor all personnel em= 
ployed to operate the instructional pregram in the learning 
centers * 



109 



ERIC 



o 





8, The Contractor agrees to provide a list of performance oh s 
j actives for his instructional program in reading and mathe- 
matics, The objectives must stipulate the individual student 
achievement level required, and the cycle and level of in- 
st ruction for which these objectives are approp 1 'ite. (See 
Section VIII, Item 2, Page 7.) 

9 - The Contractor agrees to submit a student attendance record 
daily, and report to the project director at the time a 
student drops out of the program, 

10, The Contractor agrees to report the instructional system cost 
for implementation, and projections to the project director 
on April 1, 1971 as set forth in Exhibit B, 



11* The Contractor agrees to indemnify the LEA from any liability 
for damage to the Contractor-owned property, 

12, The Contractor agrees to the responsibilities outlinted in 
the proposal and addendum and RFP as identified but not 
specifically included in this contract, 

13, The Contractor agrees to instruct all personnel employed to 
operate the instructional program in the Rapid Learning Centers 
that if they are party to information relative to the standard- 
ized test being employed by the^LEA's internal evaluator to 
determine the guarantee performance level of the Contractor, 
the individual who has learned this, information shall be im- 
mediately responsible for reporting such facts in writing to 
his project director, 

IV, RESPONSIBILITIES OF LEA 

1. The LEA agrees to schedule and initially provide to the Con- 
tractor no more than 300 students with an IQ of 75 or higher 
as measured by a locally administered intelligence test ful- 
filling the following entry criteria: (a) students in the 

1969-70 Rapid Learning Center (Phase I) program who did not 
gain one or more grade levels in reading comprehension or 
mathematics (b) seventh-grade students who are two or more 
grade levels deficient in reading and/or mathematics, and 
(c) students in grades 8-12 who are two or more grade levels 
deficient in reading and/or mathematics. If any question 
exists regarding the entry level of an individual student, 
the case must be referred within fifteen student class days 
in the project according to a negotiation procedure agreed 
upon by the LEA and the Contractor, Within fifteen days 




following referral of an individual f a meeting must be sched- 
uled between the project director and the component manager 
at which time disposition of the individual case will be made, 

2, The LEA will be responsible for ensuring that any RLC student 
enrolled and in attendance for that particular day will attend 
the specific component classes operated by the Contractor, It 
will be the responsibility of the LEA to ensure that RLC 
students attend regular school classes to the greatest extent 
possible. Specific after^school program operating hours will 
be iscablished to allow RLC students who have been absent tc 
complete the work they have missed, 

3, The LEA agrees to make the RLC student available to the Con- 

tractor for a maximum of 140 days prior to the final posttes . 
If* in fact, fewer than 140 days of instruction are scheduled 
during the period of the project for whatever reason (other 
than fault of the Contractor) , the performance guarantee will 
be reduced proprotionate to the number of days ' r instruction, 
(Example* 120 days of instruction: Guarantee t crformance 

level would be 120/140, or 6/7, of the original level,) 

4, The LEA through its internal evaluator will be responsible 
for supervising the administration and scoring of the tests ; 
and continued review and analysis of all material used by the 
Contractor in the program. 

4, The LEA through its internal evaluator will be responsible 
for supervising the administration and scoring of the tests; 
and continued review and analysis of all material used by 
the Contractor in the program, 

5, The LEA agrees to schedule RLC students to the Contractor for 
45 to 55 minutes per day par subject matter area in which the 
student is enrolled, 

6, The LEA agrees to provide office space for Contractor's on- 
site component manager. Other operational expenses such as 
secretarial help, supplies, equipment, etc,, shall be the 
responsibility of the Contractor, 

7, The LEA agrees to appropriately maintain all space to be used 

by the Contractor in the instructional program, 

■ 

PERFORMANCE REQUIRED OF CONTRACTOR 

1, The Contractor guarantees that each student in the program 
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will increase his achievement in reading and/or mathematics 
by 1*0 to 1.9 grade levels* 

2. The Contractor guarantees that each student will successfully 
pass 75% of the terminal criterion-reference items. 

3. The Contractor agrees that he shall be responsible for all 
dropouts from the RLC following the initial two weeks of 
operation. The definition of a program dropout is found in 
Section VI of this contract. 

4. The Contractor shall guarantee that the operating costs of 
the proposed instructional system will decrease as a result 
of increased student enrollment , or through efficiencies when 
applied to a target population prescribed during the perfor- 
mance of this contract* 

5* The Contractor’s instructional system utilized during the 
school year 1970-71 Phase II will be guaranteed to maintain 
the cost-effect 4 veness level demonstrated during the 1970-71 
Phase II school year if the LEA adopts and incorporates it 
under the same leasing conditions into grades 7-12 in the 
regular school system during the school year 1971-72 Phase III* 
This guarantee applies only if the LEA utilizes the Contractor’s 
complete programs operant under the same conditions as obtained 
throughout school year 197U-71 Phase XI. 

6* The Contractor agrees to train to his standards a minimum of 
ten mathematics teachers, ten English teachers, and two equip- 
ment maintenance persons from the participating school dis- 
trict’s personnel to operate the learning center turnkey pro- 
gram for Phase II (1971-72) , The LEA shall select the teachers 
to be trained. The Contractor will provide information on 
teacher training cost. 

7. The internal evaluator shall, during the period two weeks 
prior to the posttest, make a quality control check of the 
instructional materials in use in the program to determine 
whether the Contractor has fulfilled the requirements listed 
in Exhibit A. Should the quality control check indicate 
drilling of exposed items during the two-week period immedi- 
ately prior to posttesting, the Contractor shall be liable 
for the coat of a complete comparison analysis of all Instruc- 
tional ’’bits” used in the two-week period with all test items, 
and In addition shall be penalized $1,000.00 for each exposed 
item. 
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8, The Contractor shall not include in any of his instructional 
materials any exercises that are the same as the items used 
in the tests that will be used to determine how much the Con- 
tractor will be paid. The definition of l! same !f would be 
determined by the rules in Exhibit A, These rules apply only 
to instructional materials that, have been copyrighted since 
the inception of Phase XI. 

VI. METHOD OF tffiASURING PERFORMANCE 

A. DEFINITIONS 

The following definitions shall apply in the program: 

1* A student will be considered a dropout from the program if he 
or she leaves school or the program and does not reenter within 
thirty days. Exceptions to this definition are: (a) if a 

student is drafted into military service (b) if a student is 
physically or mentally incapacitated to such an extent nat 
he or she is not able to participate _n the project and attend 
school as certified by a licensed physician, or (e) other 
reasons mutually agreed upon by the project director and the 
Contractor - 

2. The starting time for each RLC student will be the first day 
the student enters the program. Any exception to this pro- 
cedure must be agreed upon by the project director and the 
Contractor, and any such agreement must be made in writing, 

3* The ending time for the instructional program for each student 
shall be the date when the final standardized test is adminis- 
tered to the student. If the student takes the January and 
May 1971 standardized tests, the latter date shall be consid- 
ered the ending date. Exiting of students who have demonstrated 
e 'eptio \ial achievement will be by the mutual argument of the 
project Uj *ctor and the EDL component manager, 

4* Actual instructional time is the net instructional time spent 
in the program. 

5, Students attending RLC 1 s will be referred to herein as student, 
VII. BASIS OF PAYMENT 

1. Determination of total payment to the Contractor will be based 
on the (a) achievement gain made by each student on the stan- 
dardized tests, and (b) extent to which each student achieves 
the final criterion-reference measure. 



2, Sever- ty-five (75%) per cent of total payment will be based on 
the results of the standardized tests, and twenty-five (25%) 
per cent of total payment will be based on the results of 
student achievement on final criterion-reference measure, 

3, Total maximum project costs of $65,788,00 are to be distributed 
as follows" 

Fifty (50%) per cent of the Fixed Charge, $19,506,00, will 
be paid the Contractor at the signing of the contract; and 
the remaining f^fty (50%) per cent, $19,506,00, will be 
paid the Contractor on or before December 1, 1970, Final 
payment in the amount of $26,776,00 will be made to the 
Contractor subject to adjustment downward based on per for- 
mance and the conditions set forth under Section -V, Item 7, 
above, and Section IX, below, on or before June 30, 1971, 

VIII, PROCEDURES 

1, Standardized tests used to measure performance will be selected 
by the project director, and approved by the internal evaluators 
from the nationally standardized tests generally available to 
the school market. The project director will have authority 
over all pre- and posttesting conditions, and will adhere to 
standard testing procedures and scoring practices as defined 

by the test publisher. He will determine when the teats will 
be given, and which forms of the selected tests will be given 
to individual students. The Contractor will not be told what 
test or what forms of the test have been or will be used for 
each student, 

2, The Contractor must submit to the project director a pool of 
criterion-referenced test items. At least five (5) times the 
number of behavioral objectives inherent in the structure of 
the system to be used must be submitted and approved by the 
internal evaluator thirty (3D) days after initiation of the 
program, 

IX, FORMULA FOR PAYMENT 

A* Student Point 



A student point is a unit of measure in the amount of $26,776.00 
divided by the total point value for the number of assigned 
students. Each student will be assigned 4 points for mathe- 
matics and/or 4 points for reading. 






Four points were selected in order to fac'litata 

tion for each student in each subject area on u ?° mp “ ta= 

75% payment (3 polnCs) for norm r ^ence teLs InTw 

(1 point) for criterion reference tests! p;,y,nem 

C omputation of Contra c tor Perfomanea Pavment- 

1. Ranges of growth per student for point assignment 



Less than ^fachifvement^n finafcriteiion- 3 PtS ‘ 

referenced measure (math) 

Up to and including ,9 years growth - % 1 P<>n ‘ l3,:y pts - 

Less than 75% achievement on final criterion- ' Pe "° lty PtS ' 
referenced measure (reading) 



Achievement Guarantee : 

1.0 to 1,9 years growth (math) 

Satisfactory achievement on final criterion- 
referenced measure (math) 

1.0 to 1.9 years growth (reading) 
Satisfactory achievement on final critarion- 
rerereneed measure (reading) 

Bonus •, 

20 or greater years growth (math) 

85% or greater achievement on criterion- 
referenced measure (math) 

2.0 or greater years growth (reading) 

85^ or greater achievement on criterion- 
referenced measure (reading) 



1 penalty pts. 



No 



isignment of 



2, Computation for final payment 



No assignment of 
No assignment of 

No assignment of 



3 bonus pts . 

1 bonus pts* 
3 bonus pts* 

1 bonus pts * 



Following point assignment for all student, the balance 
■Srfi£f eS " i " US P ' nalty pol ” ts > ^ used «“eLr- 

mine final payment to Contractor. aecer 

Penalty ; 

$26,775.00 - (Student point value x penalty pt . bal.) 



Achievement Gu arantee : 



$26,775.00 - (No penalty /no bonus) 

Bonus : 

$26,775.00 + $1.00 - (Contractor agreed acceptance for 

bonus condition, regardless of 
number of bonus points earned.) 

C * Payme nt Related to Student Withdrawal for Cau se 

If the student leaves the project lor cause, the Contractor 
will receive cost reimbursement of the $26,776.00 held in 
escrow based upon a linear proration of Contractor's costs 
up to the time of the student’s departure. The Contractor ’s 
reimbursement for the exiting student’s final performance 
and his or her performance on any interim performance objec- 
tives that have not been tested will be based upon a proration 
of the mean gain of the student’s class, up to the time of 
the student’s departure, 

TEACHER TRAINING 

Teacher training for the project will be conducted by EDL personnel. 
The teaching staff will be selected from the LEA district for train- 
ing and continued teaching activities within the learning center. 
Five lab directors and five paraprof essionais will be selected for 
training, with final approval of the Contractor and the LEA, They 
will be scheduled for a five-day, iorth-hour training period prior 
to installation of the systems. Additional teachers will be 
selected and trained concurrently to provide a corpus of trained 
specialists who will be able to continue the instructional program 
if any staff members are unable to complete the year due to extended 
illness or normal teacher attrition. The Contractor agrees to 
train 20 additional district staff members in the operation of the 
system. The intent here is to form a nucleus of trained profes- 
sionals within the Texarkana districts who can be used as resource 
teachers or staff development consultants during subsequent phases 
of the Texarkana Dropout Prevention Program, The initial training 
period will consist of five consecutive days. Training will in- 
clude the component managers all lab directors, and all parapro- 
fessionals and resource consultants (staff members to be trained) , 
The training schedule (Bee Appendix B, Contractor Proposal) will 
be adhered to during the five-day initial training period. Twenty 
hours of ongoing in-service training sessions or visitations will 
be conducted by EDL or authorized representatives . The resource 




consultants will act as consultants to lab directors as required, 
and will assume responsibility for assisting EBL teacher training 
personnel during ongoing in-service training sessions, 

XI. TEACHER ADMINISTRATION POLICY 

The success of the LEA program depends on the willingness and 
ability of the teachers assigned to the program to use the meth- 
odology. If a personnel situation develops in which it appears 
that a teacher may not be serving the best interest of the LEA 
program as mutually concluded by the component manager and the 
project director, the project director shall consider the replace- 
ment of such teacher. 

XII, DISSEMINATION POLICY 

Dissemination of information pertaining to planning, negotiation 
procedures, and interim activities related to the project will be 
mutually agreed on by project director and Contractor prior to 
its release to the public. 

All information pertaining to evaluation or test results may be 
disseminated only by the project director. Subsequent to public 
release of data and information and/or following completion of 
the present contract, the Contractor mil have the right to pre- 
pare and distribute /evaluation reports, based on released data, 
and to distribute reprints of this evaluation to interested parties. 

XIII, VISITATIONS 

Visitation privileges will be extended at the discretion of and 
with mutual agreement between the project director and the Con- 
tractor, Specified times and sites for visitation will be estab- 
lished, and made available upon request to potential visitors. 

XIV. SUCCESSORS AND ASSIGNEES 

All terms, conditions, and provisions hereof shall inure to and 
shall bind the parties hereto, their, and each of their respective 
heirs, executors, administrators, successors and assignees. Con- 
tractor shall not subcontract, assign, mortgage, encumber or other- 
wise transfer any interest in this agreement, 

XV, CONVENAN! AGAINST CONTINGENT FEES 

The Contractor warrants that no person or selling agency has been 
employed or retained to solicit or secure this contract upon an 
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agreement or understanding for a commission, percentage , brokerage, 
or contingent fees, excepting bona fide employees or bona fide 
established commercial or selling agencies maintained by the Con- 
tractor for the purpose of securing business* For breach or vio- 
lation of this warranty the LEA which have the right to annul this 
contract without liability or any discretion to deduct from the 
contract price or consideration, or otherwise recover, the full 
amount of said commission, percentage, brokerage, or contingent fee. 



XVI. EQUAL EMPLOYMENT OPPORTUNITY (Section 202, Executive Order 11246, 
September 24, 1965, 30FR 11269) 

"During the performance of this contract the Contractor agrees as 
follows % 11 

1, "The Contractor will not discriminate against any employee 
or applicant for employment because of race, creed, color, 
or national origin. The Contractor will take affirmative 
action to insure that applicants are employed, and that em- 
ployees are treated during employment, without regard to their 
race, creed, color, or national origin* Such action shall in- 
clude, but not to be limited to the following* 

Employment, upgrading, demotion, or transfer, recruitment 
advertising; layoff or termination; rates of pay or other 
forms of compensation; and selection for training, including 
apprenticeship. The Contractor agrees to post in conspicuous 
places, available employees and applicants for employment, 
notices to be provided by the contracting officer setting 
forth the provisions of this non-discrimination clause." 

2. "The Contractor will, in all solicitations or advertisements 
for employees placed by or on behalf of the Contractor, state 
that all qualified applicants will receive consideration for 
employment without regard to race, creed, color, or national 
origin 

3* "The Contractor will send to each labor union or representative 
of workers with which he has a collective bargaining agreement 
or other contracts or understanding, a notice, to be provided 
by the agency contracting officer advertising the labor union 
or workers representative of the Contractor’s coaraitraents of 
Section 202 of Executive Order #11246 of September 24, 1565, 
and shall post copy of the notice in conspicuous places avail- 
able to employees and applicants for employment* " 

4. "The Contractor will comply with all provisions of Executive 
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Order #12246 of September 24, 1965, and of the rules, regula- 
. ion:;, and relevant orders of the Secretary of Labor," 

5, "The Contractor will furnish all information and reports re- 
quired bv Executive Order #12246 of September 24, 1965, and by 
the rules , regulations, and orders of the Secretary of Labor, 
or parsuant thereto, and will permit access to his books, re- 
cords , and accounts between contracting agency and the Secretary 
of Labor for purposes of investigation to ascertain compliance 
with such ruies, regulations and orders." 

6, "In the evens of the Contractor's non-compliance with the non- 
discrimination clauses of his contract or with any of such 
rules, regulations, or orders, his contract may be cancelled, 
terminated or suspended in whole or in part and the Contractor 
may :>e declared ineligible for further Covernment contracts in 
accordance with procedures authorized in Executive Order #12246 
of September 24, 1965, and such other sanctions may be imposed 
and remedies invoked as provided in Executive Order #12246 of 
September 24, 1965, or by rules, regulation or order of the 
Secratary of Labor or as otherwise provided by law," 

7 , "The Contractor will include the provision of # 137 in every 
subcontractor purchase order unless exempted by rules, regula- 
tions, or orders of the Secretary of Labor issued pursuant to 
Section 204 of Executive Order #12246 of September 24, 1965, 

so that such provisions will be binding upon each subcontractor 
or vendor, The Contractor will take such action with respect 
to any subcontract or purchase order as a contracting agency 
may direct as a means of enforcing such provisions including 
sanctions for non-compliances provided, however, that in the 
event the Contractor becomes involved in, or is threatened 
with, litigation with a subcontractor or vendor as a result 
of such direction by the contracting agency, the Contractor 
may request the United States to enter into such litigation 
to protect the interest of the United States," 

XVII, CERTIFICATION OF NON-SEGREGATEB FACILITIES 

The Contractor or subcontractor certifies that he does not maintain 
or provide for his employees any segregated facilities at any of 
his establishments , and that he does not permit his employees to 
perform their services at any location, under his control, where 
segregated facilities are maintained. He certifies further that 
he will not maintain or provide his employees any segregated 
facilities at any 'of his establishments, and that he will not per- 
mit his employees 4 to perform their services at any location under 
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hi & control, where segregated facilities are maintained. The 
Contractor or subcontractor agrees that a breach of this certifica- 
tion is a violation of the Equal Opportunity clause in this con- 
tract, As used in this certification the term "segregation facil- 
ities" means waiting rooms , work areas, rest rooms and wash rooms, 
and restaurants and other eating areas, time clocks, locker rooms, 
and other storage or dressing areas, parking lots, drinking foun- 
tains, recreation or entertainment areas, transportation,, and 
housing facilities provided for employees which are segregated by 
explicit directive or are in fact segregated on the basis of race, 
creed, color or national origin, because of ’habit, local custom, 
or otherwise. He further agrees that (except where he has obtained 
identical certifications from proposed subcontractors for specific 
periods) he will obtain identical certifications from proposed 
subcontractors prior to the aware of subcontracts exceeding $10,000,00 
which are not exempt from the provisions of the Equal Opportunity 
clause; that he will retain such certifications in his files; and 
that he will forward the following notice of such proposed subcon- 
tractors (except where the proposed subcontractors have submitted 
identical certifications for specific time periods): 

XVIII, NOTICE TO PROSPECTIVE SUBCONTRACTORS OF REQUIREMENT FOR CERTIFICATIONS 
OF N0N-3EGREGATED FACILITIES 

A certification of non-segregated facilities, as required by the 
May 9, 1967, Order (32 FR 7439, May 19, 1967) on elimination of. 
segregated fauilities, by the Secretary of Labor, must be sub- 
mitted prior to the award of a subcontract exceeding $10,000 
which is not exempt from the provisions of the Equal Opportunity 
clause. The certification may be submitted either for each sub- 
contractor or for all subcontracts during a period (i*e. „ quarterly, 
semi-annually, or annually). 

Note: The penalty for making false statements in offers is pre- 

scribed in 18 U,S,C, 1001. 

IN WITNESS WHEREOF, the parties have caused this agreement to be 
signed in their behalf by their duly authorized representatives on the 
day and year first written above. 



CONTRACTOR 



LEA 



Edmund Zazzera 
President 
EDL/McGraw-Hlll 



Appendix G 

CONTRACT BETWEEN TEXARKANA AND REGION VIII 
EDUCATION SERVICE CENTER, 1970-71 



KNOW A!,!, MEM BY THESE PRESENTS, that, Texarkana School District #7, Miller County 
Arkansas , a public school system Incorporated in the State of Arkansas with principal 
offices at Texarkana., Arkansas, designated as Fiscal Agent for a planned “dropout 
prevention project 1 to be funded by the U. S, Office of Education, hereinafter des- 
cribed as the '‘Agent’ , for and in consideration of five dollars ($5,00), and other 
valuable consideration, receipt whereof is hereby acknowledged, does hereby contract 
and agree with the Region VIII, Education Service Center a cooperative regional edu- 
cation agency administered by the Magnolia Arkansas School District if 14, Columbia 
County, Arkansas, a public school system incorporated in the State of Arkansas and 
with principal offices In Magnolia, Arkansas, hereinafter described as the "Contractor 
as follows: 

5 « 

WITNESSETH THAT: 

WHEREAS , the Agent has received continuation of an operational grant for Phase 
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It urnHr the auspices of the IJ, S, Office of Education to conduct a "Dropout Preven- 
tion Program 11 in the Texarkana, USA, area, and desires certain technical and manage- 
non t assistance in the operation of such program, and 

WHEREAS , the Contractor is prepared to provide certain technical and management 
assistance and advice to the Agent in the operation of such program. (See Attachment 

n 

NOW THEREFORE, the parties do mutually agree as follows; 

I . Scope of Work 

The services to he performed by the contractor encompass the following areas 
of works (a) curriculum services, (b) evaluation services and (c) management support 
services * 

A. The contractor shall perform the following evaluation services: 

1, Refine and complete the evaluation design for Phase 11 by September 1, 

1970 . 

2% Develop the necessary evaluation forms, questionnaires, and instruments 
designated as the responsibility of the internal evaluator according to the 
time schedule in the evaluation design, 

3- Monitor the collection of information required in the evaluation design, 

4, Provide the project director an Interim evaluation report by February 15, 

1971, 

5, Analyze the data obtained for evaluation purposes and provide the project 
director with feedback information on the analysis, 

6, Provide the project director with all Information and/or reports deemed 
necessary for efficient operation of the program. 




7 , Present to the project director a final evaluation report by August 31, 



1971 , 



Dr, Lawrence H, Roberts will represent the contractor in the per- 
formance of the evaluation services. Dr, Roberts has his Ph,D, 
decree in Counseling and Guidance^ and has extensive work exper i- 
cnee in teaching, evaluation, and governmental work* During the 
past year. Dr. Roberts was coordinator of Programs, Region VIII 
Education Service Center, He holds membership in numerous pro- 
fessional associations including the American Psychological 
Association, American Personnel and Guidance Association, National 
Education Association, Phi Delta Kappa and similar groups* 

The estimated cost for performing the evaluation services is 



B, The contractor Shall perform the following curriculum services! 

1, -Develop and implement a pre-service and in-service training program for 
turnkey teachers. The pre-service training program will be complete by 



September 1, 1970^ while the in-service will continue throughout the school 
year 

2, Monitor and provide consultative assistance to the turnkey program through- 
out tint school year, 

3. Provide the necessary management competencies as needed in the operation 
of the turnkey program, 

4- Develop dissemination information as needed about the turnkey program for 
various audiences and/or recipients. 



fil? ,400,00 
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5. Organize a curriculum study committee and help conduct a study of the 
needed vocational education programs# The study is to be completed by June 
1, 1971# 

6# Assist in the planning and development of a grading system appropriate 
to a self pacing instructional program. An appropriate grading system is 
to bo developed by June 1 9 1971. 

7, Identify the students for the turnkey program by August 15 * 1970# 

Dr# Lewis Lemmond will represent the contractor in the performance 
of the curriculum services# He will be located in Texarkana and 
will devote full time to the services outlined under this contract# 

Dr# Lemmond has his Ph.B. degree in Supervision, curriculum, and 
instruction# He has work experience at all levels of education 
including teaching, supervision, and administration. Dr* Lemmond 
holds membership in the National Education Association, American 
Association of School Administrators, National Association of 
Secondary School Principals, Phi Delta Kappa and numerous regional 
and state professional associations* 

The estimated cost for performing the curriculum services is $9,200. 

C# The contractor shall perform the following management support services# 

1# Help develop and write a '^request prop0ssi ,, to be used in obtaining 
bide for prospective contractors by July 20, 1970* 

2* Assist in the development of a criteria by which a contractor might be 
chosen. A point system for evaluating contractor f s bid will be developed by 
August 15, 1970, 




o 

ERIC 

iminaffaHagaa 



3, Develop a list of tasks needed to Initiate and operate the learning 
center , turnkey, curriculum and Instruction, and the counseling and guidance 
components . 

4* Help as needed in the development of an information dissemination system 
for persons within the project; as wall as for those outside the project area, 
5# Assist the project director In the preparation and writing of reports and 
the continuation proposal* 

6* Assist In the development of a financial record system and correlating 
the cost information with the records system. 

Dr. Dean C. Andrew will represent the contractor in the performance 
of the management support services. Dr- Andrew has his Ph,D ( degree 
In Educational Psychology and Is presently Associate Director of the 
Region V1XI Education Service Center. He possesses considerable 
experience in teaching, research, and administration. Dr. Andrew 
is the author of several books and numerous journal publications 
in the field of education. He has conducted or has assisted in 
several planning studies involving the education, health, and 
rehabilitation fields. Dr. Andrew holds membership in the American 
Psychological Association, American Personnel and Guidance Association, 
American College Personnel Association, National Education Association, 
and several regional and state professional organizations. 

The estimated cost of the management support services is $5400,00. 
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II. Responsibilities of the Fiscal Agent 



A. Consultation 
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The fiscal agent and participants staff members shall cooperate with the 
contractors representatives, and shall make themselves available at all 
reasonable times during ordinary working hours during the period of the 
contract. They shall be willing to confer with contractor on any problems 
that ar i so, and assist in the planning and implementations of the services 
included in this contract. 

B. Information 

A he fiscal agent or his designate shall cooperate with the contractor's 
representative in providing all information essential to carrying out the 
scope 1 of work described herein, 

C, Inspection and Reports 

The fiscal agent shall have the right at all times during the period of 
the contract to inspect the work performed by the contractor, and to request 
brief interim oral or written reports of work progress from the contractor as 
may bo reasonably necessary to assure proper performance of the contract. 

Til , Period of Performance 

The services of the contractor are to commence on duly 1, 1970, and will end 

June 30, 1971. 

TV, Compensation and Method of Payment 

A, Compensation 

1, To perform the services outlined in this contract, the Fiscal Agent shall 
pay to the contractor a sum of money not to exceed $32,000, and it is to 
include all costs and expenses related to this agreement and represents pay- 
ment in full for the complete and satisfactory services noted herein, (See 





budget, attachment II,) 

2, The payment under this agreement will be made upon presentation of a 
requisition for payment by the contractor and will specify expenditures for 
the following line items: 
a* Personnel 

b. Travel 

c. Supplies, equipment, and services 

d. Overhead and miscellaneous 
Method of Payment 

1, The dates listed on the evaluation design represent deadlines for per- 
formance of various^ services except where changes in deadline dates are 
mutually acceptable to the Fiscal Agent and the contractor- 

2. Payments to the contractor shall be made according to the following 
schedule! 

n. Upon execution of this agreement, the contractor shall present a 

requisition to the Fiscal Agent for the advance €>f Mfr-o f-fch e-feofeal 
budget amount of $32,000, which is $8,000. 
h# On October 1, 1970, January 1, 1971, and April 1, 1971, the contrac- 
tor shall present a requisition to the Fiscal Agent for the advance 

whieh-is $6,400 fer 

each remaining quarter of the contract period, 

g- Upon acceptance of the final evaluation report by the Fiscal Agent, 

■ 

the contractor shall present a final requisition to- the Fiscal- Agent 
for 15% of the total-budget— amount- of— $32, 000, or §4,800, 




\ 



V* Changes and Conditions 



Changes , additions, or conditions to this contract may be made only by 
mutual agreement of the parties* 

In witness whereof, the parties hereto have executed this contract this 1 
clay of July 1970* 



WITNESSED: 



Texarkana School District No, 7 

A> 

B y \£ t£ s j •* *•* k t 

Fiscal Agent 

Region VIII Education Service Center 

^ ; w £ t- 

/Director 





Super in tend (nit / Magnolia^ 
School District No. 14 



i 





Appendix H 

CONTRACT BETWEEN TEXARKANA AND EPIC 
EVALUATION CENTER, 1970-71 



KNOW ALL MEN BY THESE PRESE ITS, that, Texarkana School Dlstx'ict 
#7, Miller County, Arkansas, a public school system incorporated in the State of 
Arkansas with principal offices at Texarkana, Arkansas, designated as Fiscal Agent 
for a planned ''dropout prevention project" to be funded by the U. S, Office of Edu- 
cation, hereinafter described as the "Agent, " for Mid in consideration of live dollars 
@5,00) and other valuable consideration, receipt whereof is hereby acknowledged, 
does hereby contract and agree with EPIC Diversified Systems Corporation, a pri- 
vate Arizona corporation with principal offices at Tucson, Arizona, hereinafter de- 
scribed as the "Contractor, " as follows: 

WITNESSETH THAT: 

WHEREAS, the Agent has received continuation of an operational grant for 



Phase n under the auspices of the U. S. Office of Education to conduct a "Dropout 



Prevention Program' 1 in Texarkana, USA, area, and desires certain technical as- 
sistance in the operation of such program, and 

WHEREAS, the Contractor is prepared to provide certain technical assis- 
tance and advice to the Agent in the operation of such program, (See Attachment 

I.) 

NOW THEREFORE, the parties do mutually agree as follows: 

1, Scope of Work to be Performed by Contractor 

The services to be performed by tne Contractor for the general purpose 



1. verifying the results of the project evaluation, and 

2, assessing the appropriateness of the evaluation procedures. 

The more specific services to be performed by the Contractor shall Include: 
(See attachment) 

1, To verify the implementation of the project evaluation design. 

2 , To review toe evaluation forms, questionnaires, and instruments required 
in toe evaluation design. 

3, To review toe monitoring of the collection of information required in the 
evaluation design and as reported by the internal evaluator. 

4, To verify toe analysis of data as gathered and reported by the internal 
evaluator to the project director, 

3, To review and report on the information and/or reports preseated by the 
internal evaluator to toe project director. 



of; 
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6, To provide tlie project director with two major Audit reports — one based 
on the Interim Evaluation Report and the other based on the Final Evalu- 
ation Report presented by the internal evaluator, 

n. Audit Personnel 

The Educational Program Auditor will be Dr. Robert E, Kroner, utilizing 
a team of support personnel from the EPIC Diversified Systems Corporation, The 
support personnel will be Dr, Terry Cornell, Evaluation Design Specialist, Mr, 
Allan Gibson, Measurement and Statistics Specialist, and Dr, Robert Armstrong, 
Project Management Specialist, The resumes of these people are presented in 
Attaenment X« 

in. Audit Sampling Technique 

Duo to the extreme need for accuracy for all testing scores utilized for 
payment purposes, the evaluation activities directly associated with the administer- 
ing, scoring, and tabulating of these data will be thoroughly monitored and all 
statistical analyses duplicated. 

Other evaluation data results will be spot-checked on a basis of not less 
than 5% of the total 

IV, Audit Plan Schedule (October 16, 1970-June 30, 1971) 

1, October 15 (or at scheduled pre-testing) — On-site Visitation 

a. Observe pre-testing procedures and conditions, 

b. Interview teachers and students in the. project. 
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2 . 



October 31 — Report to Project Director on reported evaluation activity and 
data. 



3. November 30— -Report to Project Director on reported evaluation activity 
and data. 

4. December 31 — Report to Project Director on reported evaluation activity 
and data. 

5. January 10 — On-site visitation 

a. Spot check reported evaluation activity. 

b. Interview project personnel. 

6. February 1 — Process Audit Report to Project Director 

7. March 31 — Report to Project Director on reported evaluation activity and 
data. 

8. April 30 — Report to Project Director on reported evaluation activity and 
data. 

9. May 25 (or at scheduled post-testing)— On-site Visitation 

a. Observe post-testing procedures and conditions. 

b. Interview project personnel. 

10, June 25 -(or twenty days after receiving final evaluation report) — F inal 
Audit Report x 

V. Responsibilities of the Fiscal Agent 

\ 

A, Consultation 

The fiscal agent and participating staff members shall cooperate with tee 
contractor's representatives, and shall male# themselves available at all reasonable 
times during ordinary working hours during the period of tee contract. Hey shall 
be willing to confer with the contractor on any problems that arise) and assist in 
the planning and implementing of the servioes included In this contract. 






fc 

i 




B» Information 

The fiscal agent or his designate shall cooperate with the contractor’s 
representative in providing all information essential to carrying out die scope of 
work described herein and as presented in Attachment B, Outline of Educational 
Program Auditing Procedures, U.S.O.E. 

C. Inspection and Reports 

Tbs fiscal agent shall have the right at all times during die period of the 
contract to inspect the work performed by the contractor, and to request brief 
..i nterim oral or written reports of work progress from the contractor as may be 
reasonably necessary to assure proper performance of the contract, 

VI. Period of Performance 

The services of the eontraetor are to commence on October 15, 1970, and 
will end June 30, 1971. 

VO, Compensation and Method of Payment 
A. Compensation 

1. To perform the services outlined in this contract, the Fiscal Agent 
shall pay to the eontraetor a sum of money not to exceed $7,002.60, 
and It is to include all costs and expanses related to this agreement 
and represents payment in full for the complete and satisfactory ser- 
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vices noted herein, (See budget, attachment II,) 



2, The payment under this agreement will be made upon presentation of 
a requisition for payment by the contractor and will specify expendi- 



tures for the following line items i 



a. 


Personnel 




b. 


Travel 




c. 


Supplies , 


equipment, and services 


d. 


Overhead 


and miscellaneous 



B, Method of Payment 

1. The dates listed on the evaluation design represent deadlines for per- 
formance of various services except where changes in deadline dates 
are mutually acceptable to t' > Fiscal Agent and the contractor. 

2. Payments to the contractor shall be mad# according to the completion 
of the following schedule for the following amounts! 





Date 


Activity Related to Payment 


Amount of Payment 


1. 


October 15 


On-site Visitation 


$1,400,52 


2, 


December 31 


Report to Project Director 


1,400.52 


3, 


February 1 


Process Audit Report 


1,400.52 


4. 


April 31 


Report to Project Director 


1,400.52 


S, 


May 25 


On-Site Visitation 


1,400,52 



VO!, Changes and Conditions 

Changes, additions, or conditions to this contract may be made only by 
mutual agreement of the parties. 




In witness whereof, the parties hereto have executed this contract this 



twenty-eighth day of September, 1970. 



Texarkana f^hool District No. 7 

// 



By. 



UiMk 






/U' 



Fiscal Agent 



EPIC Diversified Systems Corporation 




EDUCATIONAL BROGBAM AUDIT BUDGET 



Direct Labor Costs ; 

One Educational Program Auditor (18 days) 

One Evaluation Design Specialist (2 days) 

One Measurement and Statistics Specialist (2 days) 
toe Project Management Specialist (2 days) 



Other Direct Costs; 

Travel: 

a. Air Fare (3) Tucson-Texarkana 

b. Per Diem (5 days @ $30, 00) 



$2,700.00 

300.00 

300.00 

300.00 



516.00 

150.00 



Materials and Reproductions 300.00 

Computer Facility and Personnel (complete check and 
verification of scores serving as basis for payment 
and their statistical analyses as reported) 1,800.00 

Sub-total $6,386,00 

10% Overhead (facilities and equipment) 636.60 

Total $7,002.60 



\ 



i 



! 



i 



, 1.36 
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Appendix I 

CONTRACT BETWEEN TEXARKANA AND EDUCATIONAL 
CONSULTANTS, INC,, FOR PPBES DESIGN 



CONTRACT 
BETWEEN THE 

TEXARKANA SCHOOL DISTRICT #7 
and 

EDUCATIONAL CONSULTANTS, INC, 

This contract is hereby made and entered Into by and between the 
Texarkana School District #7, a public school district organized 
and existing under the laws of the State of Arkansas, with principal 
offices located in Texarkana, Arkansas (hereafter called the school 
district) and Educational Consultants, Inc., a private corporation 
organized and existing under the laws of the State of Georgia with 
offices located in Athens, 'Georgia (hereafter called the Consultants) 
and existing under the 'laws of the State of Arkansas, with principal 



offices located in Texarkana, Arkansas (hereafter called the school 
district) and Educational Consultants, Inc., a private corporation 
organized and existing under the laws of the State of Georgia with 
offices located in Athens, Georgia (hereafter called the Consultants) 



It is the intent and purpose of this agreement to stipulate the 
scope of the work to be performed under this agreement and to de- 
scribe the responsibilities and obligations of each party to this 
contract. 



SCOPS OF THE WORK 

The work to be performed by the Consultants is as follows i 

1, To prepare a cost reporting format to be used by the 
Technology Contractor in reporting costs to the Texarkana 
School District in connection with its performance 
contract financed under an ESEA Title VIII grant, 

2, To prepare a program budgeting format which will serve 
as a basic guide for the future implementation of a 
program budgeting system by Texarkana School District 
#7, The minimum essential design elements will include 
an outline for the district to us# in the development 
of goals and objectives, a program structure, a budget 
format including accounting forms, a chart of accounts, 
code numbers and a cost evaluation format including some 
suggested cost analysis technics. 

3, To develop a sub-program budgeting format for the experi- 
mental phase of the Title VIII grant and provide assistance 
to the district with its implementation In the 1970-71 
program, 

4» To analyze costs related to Title VIII Program product 
objectives concerning pupil achievement gains and drop- 
out prevention and to compute cost/effeetiveness ratios 



PURPOSE 




for pupil achievement gains in mathematies and reading 
for the experimental program, the turnkey program, and 
for comparable pupil groups in the regular school district 
program. 

5, To provide an in-service program on PPBE5 to include a 
maximum of three days and for not more than twenty- 
five (25) persons selected by the school district. 



RESPONSIBILITIES OF THE CONSULTANTS 

\« The consultants agree to provide consulting, advisory, 
and production services necessary to accomplish the 
scope of the work as outlined herein. 

2. The consultants agree to furnish the school district 
a report in draft form describing the components out- 
lined in the scope of the work (except the in-service 
program) , 

i 

3. The consultants will assume responsibility for all 
travel directly related to the project conducted out- 
side of the Texarkana area and all living expenses 
related to the project both in and out of Texarkana. 

4. The consultants agree to use Dr* C. W. MeGuffey as 
Project Director for this project. Dr, MeGuffey will 
give direct and continuing supervision to the activities 
of personnel involved in this project. 



RESPONSIBILITIES OF THE SCHOOL DISTRICT 

1, The school district agrees to provide suitable work 
facilities and materials for use by the Consultants 
as its staff members report for work in Texarkana. 

Such facilities shall include suitable work stations, 
calculating machines, copying services, typing 
services, and access to a telephone as the need occurs. 

2. The school district agrees to furnish needed back- 
ground and other information promptly and will assure 
cooperation of its staff members in the completion 
of this projeot. 



3. The school district agrees to furnish te the consultants 
all achievement and drop-out data needed for computing 
cost/effectiveness ratios. Similarly# all fiscal 

data required for the successful completion of the 
project will be made readily available to the consul- 
tants in the requested format. 

4. The school district agrees to serve as the inter- 
mediary in obtaining needed data from the Technology 
Contractor, 

5. The school district agrees to type and reproduce the 
final report as may be required for its use* 



COMPLETION SCHEDULE 



1. The total project shall be completed not later than 
July 30# 1971, 



2* Tentative completion dates for components of the 
project are: 



a. Fiscal reporting format 
for Technology Contractor 



December 1# 1970 



b. Program budgeting format 
for Title VIII program 

c. Cost analysis to compute 
eost/effectiveness ratios 
(15-20 days after data is 
made available) 



January 11, 1971 



June- July, 1971 



d. program budgeting format 
for sehool district 



June 30, 1971 



#, In-service program on 
PPBES 



As arranged by 
School District 



COMPENSATION AND METHOD OF PAYMENT 



For services as outlined herein, the Sehool District agrees to 
pay the Consultants the sum of nine thousand six hundred and ten 




fc ' ^*5 
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dollars ($9,610.00) * This amount shall be paid in seven (7) in- 
stallments of $1,200.00 each, beginning December 1, 1970 and on 
the first of each succeeding month thereafter for six additional 
months, and a final payment of $1210.00 upon the completion and 
submission of the final draft of the report of the PPBES format 
and the cost/ef fectivenss ratios to the School District, 

IN WITNESS WHEREOF, the parties to this contract have caused this 
agreement to be signed in their behalf by their duly authorized 
representatives on the day and year as indicated below. 

On behalf of the Texarkana School District #7. 



bate Id Trice, Superintendent 



Notary i 



On behalf of the Educational Consultants, Inc. 



Date 

Attested to byr 

Secretary-Treasurer 



C. W. Me Guffey, President 









o 
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